Reading This Blog Will Get You Laid, Scientists Say: Analyzing Bad Science Reporting
We are suckers for science. We love reading about how a cure for some disease has been discovered or that a food previously thought to be unhealthy turns out to be good for us. We don’t like the technical or esoteric stuff though, instead preferring to receive our fix in simpler and unchallenging forms, like the science stories that make it to our Facebook newsfeed. Understanding this craving, the media churns out plenty of the desired content, using newly published scientific papers as source material. Relying on media outlets for our science instead of going straight to the source, however, can be problematic for one big reason.
The media tends to absolutely botch their science reporting.
Incredibly complex science is presented in a mangled and dumbed down fashion. Data that is not meant to be taken as causative is presented as absolute fact. Results are exaggerated and misunderstood, sometimes even outright made up to an utterly laughable degree. The science reporting that we read often resemble a fan fiction version of the source.
Unfortunately, bad science reporting is ubiquitous and does not appear to be going anywhere. Understanding its mechanics though, can help us not only identify it, but further our own scientific literacy as well.
In the following, I highlight two particularly egregious examples of bad science reporting and detail exactly where they are wrong. Next, I list some suggestions for how the average reader can better their scientific literacy and how the typical media outlet can improve their science reporting. My hope is that you will be encouraged to think more critically about information presented as “science.” If you want to read the full text of any source, just click on it.
The synthesis and functional evaluation of a mitochondria-targeted hydrogen sulfide donor, (10-oxo-10-(4-(3-thioxo-3H-1,2-dithiol-5-yl)phenoxy)decyl)triphenylphosphonium bromide (AP39)
What the Paper Says
When the body is in a diseased state, cells produce hydrogen sulfide which acts protectively and helps mitigate cellular damage. In many diseases, including hypertension, obesity, diabetes, and heart attack, cells lose the ability to produce hydrogen sulfide. This means that the cells are more prone to damage or even death. Scientists have designed a new compound called “AP39” which delivers hydrogen sulfide to cells that can not produce it on their own, restoring their ability to protect themselves.
“The smell of flatulence has secret health benefits - and could help stave off cancer, strokes, heart attacks and dementia, scientists have revealed.”
“Whoever smelt it dealt it - but did you know that whoever smelt it could also have a reduced risk of cancer.”
“The smell of flatulence could help avoid cancer, strokes, heart attacks and dementia, according to scientists at Exeter University. “
“Scientists say smelling farts may prevent cancer”
“The smell of the farts can quell dementia…smelling farts could help with heart disease, diabetes, and even arthritis. The stinkier the better.”
“Study says smelling farts can be good for you”
“Farting Can Help Prevent Dementia And Make People Around You Healthier, Study Finds”
“As much as they like to complain about it, though, researchers at the University of Exeter have found that it could actually be good for your partner to inhale your stench.”
“Sniffing your partners farts could help ward off disease”
“Farts can fight strokes, heart attacks and dementia, scientists claim”
…but there is a surprising way your husband’s worst habit may actually be increasing your lifespan: His farts are boosting your immune system. “
“…a research team at the University of Exeter have discovered it is good for you to inhale your partner’s stinky farts…”
“Smelling your partner's farts will make you live longer, reveal scientists.”
“Smelling your partner's farts could be good for you”
What Went Wrong
Researchers often issue succinct press releases discussing their findings in order to stir the interest of the media. The press release for this paper made the grave mistake of stating that hydrogen sulfide “may smell of flatulence.” It was this unfortunate wording that seemed to spawn the maelstrom of clickbait faux science reporting listed above. Though the press release was later edited to clarify the findings, it had no effect, as too many reporters had already smelled blood (or farts.)
Yes, the press release could have been better worded, but I strongly believe the onus for the terrible reporting it inspired lies with the journalists themselves. Had any of the writers carefully read the press release or, god forbid, the source papers, it would have been abundantly clear that the study had nothing to do with flatulence and even less to do with smelling it. Beyond the brief mention in the press release, there is not a single use of the words “fart,” “flatulence,” or even “smell” in the source papers themselves. Another brief scan of the material would also make apparent that not only were all the studies done on rats as opposed to humans, but that the authors specifically mention further research being needed to verify their findings.
The notion that smelling farts or farting itself is in any way good for you is ludicrous, but perhaps the misunderstanding can be partially chalked up to the authors of the stories reading the questionable wording of the press release and running with it. No such benefit of the doubt can be given, however, in regards to the specific health claims made by the stories. Not only do the press release or the source papers not make the aforementioned health claims, there is literally not a single mention of cancer, dementia, heart disease, diabetes, stroke, arthritis, or lifespan in the press release or the papers. The authors seemed to have completely made them up. It is with this complete and utter fabrication that the reporting goes from playfully ignorant, misappropriating the findings and claiming that they had anything to do with smelling farts, to downright malicious, claiming that smelling farts will cure various diseases. Reporting such drivel moonlighting as real science exploits readers sympathies and hopefulness in a sickeningly parasitic manner. For instance, though many of the outlets placed their coverage in their “health” section, “The Mirror” took it a step further and plunked it in the “cancer” subsection. The story can be found on their website, tastefully sandwiched between two videos, one of a nurse singing to a child undergoing chemotherapy, and another of a high school student speaking to his class about his recent cancer diagnosis.
Though much of the reporting was done by websites particularly prone to clickbait (LadBible, Uproxx) and newspapers that are actually closer to tabloids (The Mirror, Daily Mail, New York Post), I was disappointed to find that several otherwise reputable outlets jumped into the fray. “Time” for instance, originally ran the article with the title “Scientists Say Smelling Farts Might Prevent Cancer.” Though they later heavily edited the article, the original title is still in the link to the current article, and because the internet is forever, the original article was easy to find it was unfortunately among the worst of the bunch.
The aspect of this entire escapade that I found most peculiar is that several of the reports specifically mentioned smelling “partners” flatulence as beneficial. Hilariously, “Readers Digest” placed their story in the “relationship” section.
Improvements in skeletal muscle strength and cardiac function induced by resveratrol during exercise training contribute to enhanced exercise performance in rats
What the Paper Says
This experiment aimed to determine the relationship between resveratrol supplementation and several metrics relating to exercise performance in rats, including fatigue, cardiac level, and skeletal muscle strength. To do this, the rats were split rats into 4 groups.
Group 1: No previous exercise training (out of shape rats) and no resveratrol supplementation.
Group 2: No previous exercise training + resveratrol supplementation.
Group 3: Previous exercise training (in shape rats) and no resveratrol supplementation.
Group 4: Previous exercise training + resveratrol supplementation.
The rats that were given resveratrol were given the equivalent of 146mg/kg of body weight each day.
After the training and resveratrol supplementation periods, the rats ran on a treadmill until exhaustion to test exercise performance. The study found that resveratrol did indeed improve exercise performance. The rats with previous exercise training and resveratrol supplementation did best, followed by rats with previous exercise training and no resveratrol supplementation, followed by rats with no previous exercise training and resveratrol supplementation, and the rats without either did the worst.
“A glass of Red Wine is the Equivalent of to an Hour at the Gym, Says New Study.”
“A glass of Red Wine is the Equivalent of to an Hour at the Gym, Says New Study.” (Note: this article copied the exact title of The Huffington post article and the body nearly word for word)
“A new study suggests that a glass of red wine could mimic the benefits gained by going to the gym for an hour.”
“Too busy to lace up your sneakers and head for a spin class? Relax…you might get the same benefit from popping open a cabernet, pouring yourself a glass, and sitting back and enjoy the same benefits as a workout.”
“Canadian researchers say a glass of red wine could have the same benefit of an hour's worth of exercise…”
What Went Wrong
With the coverage of this study, we once again see respectable and important science reduced to a pile of smoking clickbait rubble. The notion that drinking wine is “as good as an hour at the gym” demonstrates an incredible misunderstanding of the experiment and its findings in several ways. First of all, the figure of one hour at gym seems to be have pulled out of thin air because there is absolutely no way to derive that from the source. Secondly, and this should be obvious, wine ≠ resveratrol. Yes, resveratrol is found in wine. To consume the daily amount of resveratrol that the rats were given, however, a 160 pound human would have to 124 liters of wine in a single day.
Slightly less egregious, but as unforgiving for a journalist who only needed to glance at the (free) paper, is the gross misinterpretation of the results. The data does not suggest that resveratrol alone significantly effects exercise performance and related metrics. Rather, it suggests that resveratrol along with exercise effects performance.
As previously stated, each group of mice ran on a treadmill until they ran out of energy as an endurance test. These were the results for how long each group was able to run on average.
“Out of shape rates” without previous resveratrol supplementation = 30 minutes
“Out of shape rats” with previous resveratrol supplementation = 40 min
“In shape rates” without previous resveratrol supplementation =100 min
“In shape rates” with previous resveratrol supplementation = 120
So again, yes the resveratrol did help overall, but the data shows that simply exercising improves endurance more than just sitting on your couch and eating resveratrol/ chugging wine as the reporting seems to suggest. Even a quick scan of the paper makes this clear, as the fifth line states : “Our results establish that resveratrol is an effective ergogenic aid that enhances exercise performance over exercise alone.” Additionally, like the last paper, the experiments were done on rats, not humans, which means that the results can not be assumed to be equivalent in humans without further testing.
How to Not Get Duped by Bad Science
Due to either deliberate unscrupulousness on the part of writers or just willful ignorance, we are absolutely bombarded by this kind of bad science reporting on the internet. The examples listed above are particularly flagrant but other bad science reporting can be much more subtle and therefore harder to recognize as faulty. When attempting to discern good science reporting from bad, it is helpful to consider the following points:
Realize that a large amount of medical and science studies that get reported on are done on animals, rats and mice especially, and most won't make it to human testing or to market. A drug that is effective in rodents could very easily not work or even kill a human. A drug working in rodents is a good sign, but a story that extrapolates this to humans is faulty.
Avoid clickbait peddlers. Though an unfortunate amount of news outlets have resorted to catchy headlines and generally irresponsible reporting, some are worse than others. Websites that derive traffic mostly from Facebook (Uproxx, LadBible) are recurring offenders, as are lower integrity newspapers (The Mirror, Daily Mail, and the New York Post) Avoid reading articles from these sources.
Even if the reporting appears to be accurate, it is still incredibly helpful to read the source paper. Scientific papers are often incredibly complicated and sometimes need to be bought, but a good understanding of the science can be had just by reading the first paragraph of the paper, the “abstract,” which serves to succinctly sum up the findings.
These articles sometimes quote the original study or press release to back up their bad science claims. Typically though, the quotes are misappropriated and do not actually confirm what is being stated. Make sure to read the quotes carefully so you know if they are being used properly or just to make the article sound more scientific.
How Science Reporting Can Be Improved
I completely understand the current state of the journalism industry and how hard it is to stay afloat. I also understand that to get readers, outlets must often resort to dumb clickbait tactics. That, however, does not excuse the complete lack of accuracy endemic in science reporting. Assuming an outlet wishes to accurately convey scientific findings, the following suggestions may prove helpful.
Have someone with a science background do the science reporting. They do not need to have a PHD, even an undergraduate degree would serve. If the writer has even a basic understanding of science it would help ensure that the stories are more accurate.
Read the actual paper. In the aforementioned examples, I find it incredibly hard to believe that any of the writers did so much as take a casual glance at the source papers. A careful read of the entire article would be ideal, but even just reading the abstract may suffice. Because many papers need to be bought, outlets should consider buying subscriptions for a database.
Give your readers the benefit of the doubt. It should be assumed that readers lack a thorough understanding of the science, yet are curious and possess the capability to grasp it if properly explained. Though complicated, many scientific papers are incredibly interesting. If broke down properly, they can be reported on in an engrossing manner without the need for clickbait tactics.