A generative data-visualisation of all the scientific evidence for popular health supplements by David McCandless and Andy Perkins.
I’m a bit of a health nut. Keeping fit. Streamlining my diet. I plan to live to the age of 150 in fact. But I get frustrated by constant, conflicting reports and studies about health supplements.
Is Vitamin C worth taking or not? Does Echinacea kill colds? Am I missing out not drinking litres of Goji juice, wheatgrass extract and flaxseed oil every day?
In an effort to give myself a quick reference guide, I dove into the scientific evidence and created a visualization for my book. And then worked with the awesome Andy Perkins on a further interactive, generative “living image”.
Play with interactive version | See the still image
This visualisation generates itself from this Google Doc. So when new research comes out, we can quickly update the data and regenerate the image. (How cool is that??) Hopefully then this should be a useful web resources for years to come.
About the image
This image is a “balloon race”. The higher a bubble, the greater the evidence for its effectiveness. But the supplements are only effective for the conditions listed inside the bubble.
You might also see multiple bubbles for certain supps. These is because some supps affect a range of conditions, but the evidence quality varies from condition to condition. For example, there’s strong evidence that Green Tea is good for cholesterol levels. But evidence for its anti-cancer effects is conflicting. In these cases, we give a supp another bubble.
The evidence
We only considered large, human, randomized placebo-controlled trials in our data scrape – wherever possible. No animal trials. No cell studies. Many of the health claims made by the $23 billion supplements industry are based on non-human trials. We wanted to cut through that.
This piece was doggedly researched by myself, and researchers Pearl Doughty-White and Alexia Wdowski. We looked at the abstracts of over 1500 studies on PubMed (run by US National Library Of Medicine) and Cochrane.org (which hosts meta-studies of scientific research). It took us several months to seek out the evidence – or lack of.
You can see our key results in this spreadsheet. (It’s the same spreadsheet that generates the interactive image).
Generation Game
This is our first interactive piece here on Information Is Beautiful. We’ll be doing more generative pieces over the next few months so stay tuned!
Updates
UPDATE 1: 6th Mar 2010 - We’ve updated the spreadsheet (changes in bold) and the viz after great feedback and new evidence and corrections from our amazing visitors. Thank you all so much.
Quick summary: We’ve added CoQ10, cocoa, capsaicin, L-Lysine and hyaluronic acid to the chart. And adjusted entries for Magnesium, Tumeric and Omega 3 (part of a massive downgrade of Omega 3 in fact). Read the change-log for full details
UPDATE 1: 10th Mar 2010 - Omega 3 has been downgraded in almost every category. See the spreadsheet (changes in bold) for more details.
See the visualisation | Check out the spreadsheet
Feed Us Back
As ever, we welcome your thoughts, crits, comments, corrections, compliments, tweaks, new evidence, missing supps, and general feedback. Thank you!
252 Comments
What a great representation of information a lot of people want and need, I am glad I “stunbled upon this one”
This post is so COOL! I love the “ballon race” that shows the effectiveness of health supplements/products. Can’t denied that Green Tea is good for health. I drink them everyday.
That is amazing! You used the Cochrane reviews, clearly… they have an incredible set of reviews on the care of pregnant and laboring women. It would really help families make good decisions about their health care, and doctors make better decisions about their routines and protocols, to see a chart like this on medical interventions and other aspects of care. Like ultrasound, denying food and drink in labor, walking to induce labor, etc.
Perhaps the size of the bubble could show its balance of risk to benefit; larger bubbles more helpful/ harmless. The book A Guide to Effective Care in Pregnancy and Childbirth by Enkin et al summarizes the Cochrane database on this topic, and even has a series of charts in the back that could easily be turned into an interactive chart like this.
Whaddaya think? Heck, I’d even be willing to do it myself, if I could figure out how you did it…
Nice work!
I believe “polphenols” should be “polyphenols”.
Please consider adding Meltonin both for sleep and cancer support. It is widely used and has a lot of data to back it.
Many kind thanks for your considerable efforts– your collation serves as a credible reference, yes, but it’s best used as a springboard to our own individual research.
A couple of suggestions that might warrant inclusion on your list:
1. GABA, for stress/anxiety. Efficacy via oral admin has been controversial, as GABA is not believed to cross the blood/brain barrier. However, here’s an abstract (human clinical) that suggests otherwise:
http://www.ncbi.nlm.nih.gov/pubmed/16971751
2. L-Carnosine. At present, I’ve no idea if any human trials have been conducted, but I figure you guys’ll dig up the latest dirt.
http://www.ncbi.nlm.nih.gov/pubmed/20552048
Carnosine’s ability to react with MG and possibly other deleterious carbonyl compounds, and scavenge various ROS, may account for its protective ability towards ischemia and ageing.
I agree that melatonin should be added for sleep. I’m not sure about cancer… I think I read there’s a slight decrease on onset of certain types of skin cancer?
Regardless, please keep updating this!
This is pure genius! Think of the applications for presenting data in a more visual format. The fact that you can generate this from Google docs. frankly amazes me. What kind of code do you use to generate the image?
I am an omega 3 person myself. And, I have also stumbled upon the ganoderma / reishi mushroom. I am all for natural herbs and plants.
Love it – any data on rhodiola?
Really, Omega 3 Supplementation for depression is below the “worth it” line? Omega 3 supplements have received empirical evidence regarding long-term effectiveness that is at least comparable to the long-term effectiveness of any antidepressant medication. Combined with other lifestyle changes, including exercise, improved sleep hygiene, and increased time spent socializing, omega 3s easily beat antidepressant meds in the long-run.
Sincerely,
A Clinical Psychology Ph.D. student (one year from internship and two years from attaining the Ph.D.)
Its great representation but I couldnt find ‘Resveratrol’ in it. I really wanted to see how that did it with the hype and evidence.
Beautiful!
Can you please add the date of the latest update (including the year) to the visualization?
Thx!!
What about Grape(fruit) seed extract (GSE). I have been told to use it for boosting immune system while traveling etc… The people I traveled with (and myself) found it useful but that may have been more placebo than anything else (either way it worked). Are there any studies on it?
Thanks for being such a health nut that drives you to present this data so beautifully and usefully. But to keep healthy is not so good to be nut, so take it easy. Apart from nutrition, a peaceful mind in needed to reach a long life.
Another ingredient for your kind consideration: cordyceps sinesis
Long live Information is Beautiful!
Why is beta glucan orange (low evidence + promising results) but it’s balloon is in the ‘strong evidence’ range at the top of the graph?
ps. sorry i forgot to mention that i think this is just the most fantastic idea! :)
Sleep and relaxation are the tip of the iceberg where lavender oil is concerned, it has anti viral, and bacterial qualities amongst other things and is excellent on burns. I don’t have links to the research and don’t really have the motivation to find them but I use it extensively and am concerned that its position on this chart is misleading your public.
It is well known that a small amount of lavender will relax and promote sleep whereas a larger dose is a stimulant, also different types have different effects, Lavender spike is not a relaxant.
I appreciate what you are trying to do here but fear that it oversimplifies the issues which also vary with individual constitution.
Sadly its not working! I saw it on a TED video! Hopefully its back up soon :(
It’s been under construction for some time now… let us know your proposed timeline for getting it back up, it’s too useful a resource to let it wither away.
Was the current information so hopelessly incorrect that you had to remove it from the website while in the process of updating it? Otherwise I don’t understand why this has been down for so long. It’s not like it’s a physical object that you have to remove in order to work on it. How about leaving a copy here while you make the updates that apparently take more than a month to complete? Not to be a whiner, of course, but I really miss this.
What a useful way to display this data! I’m wondering when the interactive version is going to be available again – I’ve only seen the static image.
It’ll be up again this week!
add Alpha Lipoic Acid
I wish you were more transparent about how you compute the “strength of evidence” statistic. Is it simply the number of studies on each side of a debate? How do you decide if a group of studies supporting a hypothesis is a “6″ as opposed to a “5″? Does it weight studies with methodological errors (improper controls, low n, etc) less heavily than well constructed studies? How do meta-analyses factor in? Does the statistic account for biased samples? (for instance, the study supporting cognitive enhancement for creatine puts this hypothesis above the “worth it” line with a score of “4″ but uses “45 young adult, vegetarian subjects” and the authors note that the benefit may come from compensation for lower creatine levels in young vegetarians (see Discussion section) – so although it seems that way from a cursory look at the visualization, this evidence does not reflect a benefit for the general population!) Really, a visualization is only as useful as the rigor of the data it’s based on. . . so to avoid misleading people, the “strength of evidence” statistic needs to reflect the unfortunate realities of scientific literature – that not all studies are created equal – in a systematic way not prone to bias.
Ref:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1691485/?tool=pubmed
Fabulous idea. what types of studies did you include from Pubmed? Was it all inclusive or only RCT’s?
I thought there were large-scale tests regarding chocolate as a cough suppressant? Certainly a lot nicer to use than licorice root.
Neat idea, but there are a couple of problems. Licorice root is listed as having a positive antiviral effect, but it links to a study about indigestion, and that study involved a cocktail of herbal extractions rather than just licorice. Second, and involving more than just data validation, is that the value of a supplement is apparently based on a single study. Addressing this will require a more complicated underlying database, as well as an N-dimensional balloon race (or a race of N-dimensional balloons, if that’s easier to visualize).
Nice interface though. I just wish science were this simple.
Ah, excellent! The graph is so much more navigable in this interactive format. Kudos to David and Andy.
I should mention that the paper linked from Melatonin (Dodick & Capobianco, 2001) is about cluster headaches which is a medically distinct condition from migraine. Treatment for the former may be irrelevant to the later as they (may) have different causes. Just giving you the heads up!
May I ask what language was used to create this visualization?
This is brilliant!
Hello
The interactivity, animation and design is lovely.
I am a health enthusiast and have experimented with many supplements and healing modalities over the last 10 years for my own ailments and digestive problems.
You present your material well, but rather than taking statistical information for your presentation I challenge you to be brave and actually print what works for you and ask your readers to be able to somehow contribute to it. Scientific studies and political agendas aren’t mutually exclusive, and even double blind trials don’t test against different metabolic types/blood types and doshas etc. The best test is to test yourself. Its the only way you can prove anything works. Though it may be nearly impossible for institutions to control such tests, you can test yourself and you can ask your readers to test themselves and ask for genuine responses.
Meanwhile, I suggest reading David L Watts – Trace Elements,
best wishes
I absolutely love the visualisation, but it was a shame that the liquorice root bubble on the anti-viral page is labelled as being useful for coughs, but links to a study of a multi-ingredient supplement and its effects on dyspepsia.
There is a link between both acid and non-acid gastro-oesophageal reflux disease (probably mediated by vagal stimulation) but this does not seem to be the major focus of the paper linked to. Is it that this is one of many studies?
Clicking on “the change log” gives an error message.
By the way, Consumer Reports magazine recently published an article on supplements in the Sept. issue. There’s limited overlap but good agreement where they do.
“gluthathione” should be glutathione.
I suggest dividing the “none” category from the “slight” category with a line like the worth it line. It’s very hard to tell what has little evidence, and what actually has none. This distinction, IMO, is the most important in the list.
Also, I was hoping to see vitamin D3 in prevention of numerous health problems such as the flu. I’ve seen some studies, but haven’t looked into it enough to know the credibility of those studies.
I think the evidence on saw palmetto for urinary is pretty solid, maybe you can add that next update.
Of course, many scientists would argue that your being convinced of the supremacy of large double blind studies is itself misguided. Big studies make for good statistical data sets, so modelers prefer to use them. But, these are precisely the studies that suppress the value of certain treatments (medical AND nutrient-based) for outlying and peripheral members of the study group. Each of your balloons should carry a large asterisk with the warning “this analysis probably does not apply to you unless your circumstances by chance represent the average person in the study/metastudy we chose to include.”
When “evidence” is based only on statistically significant effects on a randomized group, it does not capture the signficance of a treatment that would be particulary efficacious to a particular person at a particular time. Rather than rely on the fictitious value of grand averages, I would prefer to focus on the particulars of an individual’s needs (both pyschic and physical). This requires deep scientific and affective knowledge of a subject that a randomized double blind study will never be able to capture.
Nice visuals though. How do we get hold of these tools to make representations of our own data?
As much as I love the presentation I am even more intrigued by the comments afterward. You obviously challenge many beliefs. Hrrumph! One of your responders mentioned she’d like to see the same type of balloon graph for pharmaceuticals and other medical interventions. . Any chance of reviewing Cochrane evidence for the top 100 prescribed drugs with an evidence of harm or NNT overlay?
Does anybody have any info on L-Glutamine?
cannot seem to find it here.
Thanks.
I love the graphic but was a bit distracted by a couple of points
(1) The X axis represents nothing in particular, correct? There is no significance to whether a bubble is more right or left just up or down? I spent some time looking to see why Vit. D was so far right for gen health… it just seemed to beg for some meaning. Number of citations maybe?
(2) How exactly do you compute “strength of evidence”?
(1) X axis is alphabetical order – or else it would be too difficult to find a supp
(2) we are in the process of revising our methodology – will share once we’ve nailed the new approach
thanks!
D
Your link for licorice root (for coughs) goes to the wrong study
The green tea/cholesterol bubble really caught my eye, however, there is one problem, and that is the study does show green tea reduces cholesterol with a high degree of confidence. However, the amount green tea actually reduces cholesterol is almost negligible.
I have written a post about this with full graphs here:
http://www.healthaliciousness.com/blog/How-much-does-green-tea-lower-cholesterol.php
You even state:
“For example, there’s strong evidence that Green Tea is good for cholesterol levels.”
There is strong evidence, however, it takes 10 cups a day to reduce your overall cholesterol level by 2% or so! Not exactly the key to cholesterol problems, though it is nice to know every bit helps, it seems misleading that green tea should get such a large bubble.
brilliant – thank you Paul
Difficult to include, maybe, but another source of information that I think would be helpful in judging the efficacy of supplements is who funded the research. The more I study science the more I realize that results of studies can be interpreted in many different ways depending on the motivations of the researchers. Perhaps snake oil purveyors fund dubious research to increase profits on supplements, or perhaps pharmaceutical companies want to downplay the efficacy of some supplements as compared to much more profitable synthesized drugs. I’m sure it goes both ways.
Harder to include would be the quality of the research being done. Shoddy research, including simplistic does X supplement help Y condition questions with little in the amount of control seems to be the norm coming out today. I don’t believe that many of these supplements work in such a straightforward way. In addition, individual’s body chemistry is highly variable and a supplement that is effective for one person may not be effective for another person with the same presenting condition.
Still, excellent design :)
Love this – thank you.
I can’t see aspirin or salicylic acid. Is it there?
Lots of stuff coming out about having anti-cancer properties.
Any comment?
recheck you google doc source for red yeast. I think you meant to put lowering cholesterol not high blood pressure. Awesome graphic. Thanks.
I read John Walker’s personal regimen at http://www.fourmilab.ch/documents/health/supplements/
What about the following ones cited by Mr. Walker but not in your list?
Aniracetam
CDP Choline
Centrophenoxine
Nattokinase
Piracetam
Vitamin K
Zinc Glycinate
You know what would be great? Is if you could do another one by health problem- so if someone has high cholesterol, they could see what supplements would help without having to scroll over every bubble.
Are you taking into account the fact that a supplement made in a lab yet given the same scientific name as its natural counterpart is typically the one not “worth it” because the body will not absorb it as effectively as it will absorb a supplement that comes directly from the natural source itself?
Lovely graphic, but it’s off on what may be the central use case.
For instance, if I’m concerned about what might work for cancer, I click on Show Me->Uses->Cancer, and the graphic highlights Calcium, Selenium, and Vitamin D. So far, so good.
But I also see about twenty more tiny bubbles in the good zone, and what they’re highlighting – all substances that I might care a lot about – is obscured because the bubbles are so small. The bubble size correlates to Google popularity, but for this case, I don’t care. Even if no one searches on on “Statins”" (the good bubble just to the right of Selenium), I’d be happy to know about it.
I would class this as a partial fail – the graphic conventions obscure useful information.
Hmm – where’s marijuana? Not up there because it’s (often) not legally available?