This is part four of a series on posts on fact checking science related articles.
Digging Through a Scientific Study
One of the very best ways to fact check an article is to read the original scientific study. While science journal articles can be tremendously complex, with a few hints and a little scientific knowledge, you can usually pick out enough to know how accurate an article about the study is.
- The first step is to find the original study. Sometimes blogs and news articles will have a link to the original document, but often it is up to you to track it down.
If you can find the name of the original study, search for it on Google Scholar. You may need to put the name of the study in quotation marks to narrow down the results.
- Once you find the article, you may find that you have to pay for access. Be prepared. Most articles are not cheap. Before you spend your money or give up, look at the Google Scholar listing. At the right you may find links to versions of the article that have free access.
- If you can’t get access to the article, you can still read the abstract, and often that has information that is useful in fact checking. Even if you do have full access, the abstract is a great place to start. It tells you what the study is about, and usually some very basic information about how it was conducted.
- Don’t let all the technical jargon scare you away. You don’t have to understand all of that in order to do some basic fact checking. Refer back to the article you are fact checking, and compare it with the abstract.
- Do a little research on the journal that published the article. There was a time that being in a peer reviewed journal carried a lot of weight, but more and more “anything for a buck” journals are popping up. Many of them will publish almost anything, as long as you pay a hefty publishing fee.
- While you are in a research mode, do a quick internet search for the author(s). Is the topic of the article their specialty? If you are reading an article on plant genetics, you would hope that the author was an expert in plant genetics, not a computer programmer. Not being a recognized expert does not mean they are not credible, but it does mean you should look carefully.
- Do an internet search for the name of the study, plus the word “review.” This will give you an idea of what other scientists say about the article. Be sure not to just read the reviews that agree with your position. I usually try to read at least three positive and three negative reviews to get a feel for the study. Look for reviews that are from experts in the field, as they are most likely to be accurate. if almost all of the reviews seem to agree on the quality of the study, that is a good indication.
- Check the sample size. Skim down to the part of the paper that describes how they conducted the experiment. Look carefully for how many things were tested. For example, if you are reading a study claiming that coffee cures cancer in laboratory rats, look to see how many rats were in each of the test groups. The more test subjects the have, the better. If the study only used 10 rats, it would not be nearly as convincing as if it used several hundred.
- Read the results and conclusions. Most well written papers are very open about potential problems with the study, things that could be done to improve the tests, and how conclusive the data was. Overstating the conclusions is the most common problem that I find when fact checking a science article. If the author of the original study says that the data was “promising and needs more study”, that does not mean “This study provides conclusive proof.”
How to Read a Scientific Paper
OK, if that did not sound too difficult, would you like to give it a try? First, our example is a real paper from a real science journal. I picked it because it is a well written paper about a subject that is often the subject of misinformation.
Imagine that you have just read an article from a health and fitness site saying that a new study proves that high fructose corn syrup causes diabetes and a variety of endocrine and metabolic conditions. You do an internet search and find the article in the journal from the American Society for Clinical Nutrition. The following link will open that article in a separate window, so you can follow along.
A Quick Background Check
A little research will show the American Society for Clinical Nutrition to be a very professional, peer reviewed journal. If you click on the author’s names, you can see some of their other publications. Clicking Author Affiliations will show you where they work, and what they do. Both are professionals in fields that relate to the topic, and neither seem to have any affiliation with pro-high fructose or anti-high fructose organizations.
Scrolling down almost to the bottom of the page, we see a list of other journal articles that cite this one. This article has been cited by many other articles in professional, peer reviewed journals, which is a very good sign that the science is good.
Now lets look at the abstract and the introduction. This article is a review of a previous study the authors conducted, describing the experiment, and their results. Notice the following:
“Results from a study conducted in rhesus monkeys produced equivocal results. Carefully controlled and adequately powered long-term studies are needed to address these hypotheses.”
There were no solid conclusions from their rhesus study.
“Data from a short-term study comparing consumption of beverages sweetened with fructose, glucose, high-fructose corn syrup, and sucrose suggest that high-fructose corn syrup and sucrose increase postprandial triacylglycerol to an extent comparable with that induced by 100% fructose alone.”
What is the world is postprandial triacylglycerol? A quick internet search shows that postprandial means "after eating". Triacylglycerol, sometimes known as triglyceride, is a substance that your body uses to store energy. An increase in postprandial triacylglycerol means that after eating, the Triacylglycerol level in your blood increases. The study found that high-fructose corn syrup, sucrose (the sugar we use for cooking, etc.), and 100% fructose all produced similar elevation of Triacylglycerol.
The Details of the Study
Scrolling down to “FRUCTOSE AND LONG-TERM ENERGY BALANCE IN RHESUS MONKEYS”, we can see the sample size, in this case, the number of monkeys they used. They had 16 adult, male monkeys. Sample size is tremendously important in scientific studies. The larger the sample size, the more convincing the data is. This is a fairly small sample size, indicating that the study should be replicated by other scientists to add more data.
All were fed the same diet, except for sweetened beverages. Eight were given beverages sweetened with fructose, and the other eight were given beverages sweetened with glucose.
Here we need to do some more quick research.
- Fructose (C6H12O6) is commonly found in honey, fruit, berries, and root crops. It is much sweeter than other common sugars, and is more than twice as sweet as glucose.
- Glucose (C6H12O6) has the same chemical formula as fructose, but the structure is different. giving it different properties. This is the sugar formed when your body digests starches.
- Sucrose (C12H22O11) is the sugar we commonly use in cooking. When digested, it breaks down into glucose and fructose. Cane sugar, beet sugar, date palm sugar, sorghum and maple sugar are all sucrose.
- High-fructose corn syrup is a mixture of fructose and glucose. Different varieties have different proportions. HFCS 55 is 55% fructose and 42% glucose. It is the variety used in soft drinks. HFCS 42 is 42% fructose and 53% glucose. It is commonly used in baking. You will notice that they don’t add up to 100%. The balance is made up of other chemicals, generally starch and other carbohydrates. It may be interesting to note that honey has the same ratio of fructose and glucose as HFCS 55.
OK, now that we know about the different sugars, lets go back to the article. Still in the same section, notice the following:
“Monkeys fed fructose beverages gained significant amounts of weight at 3 and 6 mo compared with their baseline weights, whereas the animals consuming glucose did not. However, weight gain was not significantly different between the 2 groups by the end of the study at 12 mo. “
At times, the monkeys drinking the fructose gained more weight, but in the long run, weigh gain was about the same for both groups. If you quoted just the first sentence, without the second, you could make it appear that this study provided more negative data about HFCS than it actually does. This is why it is always important to read quotes in context of the original article.
Continuing down to “FRUCTOSE AND LIPID METABOLISM”
and scrolling down a bit, we find the following:
“In these studies, we have compared the metabolic effects of beverages sweetened with fructose and glucose alone; however, pure fructose and pure glucose are not commonly used as sweeteners.”
“It is reasonable to hypothesize that the endocrine and metabolic effects of HFCS and sucrose would be similar to each other and that both would produce responses intermediate between those of pure fructose and glucose, and we have investigated this hypothesis. In a short-term study comparing the effects of consuming beverages sweetened with HFCS, sucrose, fructose, and glucose (25% of energy) with meals in male subjects, consumption of either HFCS- or sucrose-sweetened beverages produced postprandial glucose, insulin, and leptin profiles that were intermediate to responses induced by pure fructose and pure glucose. However, unexpectedly, postprandial triacylglycerol responses to consumption of sucrose and HFCS were comparable to 100% fructose in both peak concentrations and integrated 24-h areas under the curve.”
So they found that sweetening beverages with HFCS, pure fructose, and sucrose (regular table sugar) all produced the same results. This was surprising to the authors, which makes this important data. In science, surprises mean an opportunity to learn something new.
What Does It Mean?
The study found evidence that consuming fructose sugar increased Triacylglycerol levels more than consuming glucose. Since Triacylglycerol have been linked to heart disease and strokes, this is important data. They also found evidence that consuming sugar that is a blend of fructose and glucose caused a similar rise in Triacylglycerol to consuming 100% fructose. That means that table sugar, honey, palm sugar, agave nectar, high fructose corn syrup, and other similar sweeteners produce similar increases in Triacylglycerol. The authors noted that some of their results were equivocal, and that more study was needed.
Many people will probably be surprised that this study lacks definite conclusions and absolute evidence, but you won’t find either in single, scientific studies. To reach solid conclusions requires many related studies, some replicating the original, and others exploring other approaches to the subject.
Think of each study as a single brick used in the construction of a brick wall. The more bricks (scientific studies) you have, the larger and stronger your wall (conclusion) will be.
So don’t be overwhelmed by the complexity of scientific papers and articles. Dig out the important bits, and learn to compare the original with what blogs, news media, and other sources are saying about it.
Knowing the science is always the best form of fact checking.