Diabetes Self-Management Blog

The recent ACCORD fiasco gave a good example of how science can be used to twist people’s thinking and public policy. If we want to make good decisions in a world full of people trying to deceive us, we can benefit from learning how science is supposed to work. That way, we can recognize when it is being misused.

The scientific method
The scientific method was one of the great advances of human history. Instead of just thinking about the world and guessing what makes it tick, science insists on collecting information (data) through observation and experiment. Then, you formulate a theory (called a “hypothesis”) about it. What produced the results that you observe? And you test that hypothesis with more experiments, to see if it holds up, and modify it as necessary.

These steps must be repeatable by others. Almost any result can come up one time because of experimental error or a freak occurrence. If you don’t write everything down and share it with others, so that they can confirm your results, it’s not really science.

Understanding experiments
There are many kinds of experiments. The one most used in the health-care field is called the “randomized controlled trial,” or RCT. In an RCT, people are assigned at random to one group out of two or more. These groups should be as similar as possible and should receive the same treatment except for one factor, often a drug. Everything else is “controlled” for, meaning everyone is the same. So whatever differences you see between the groups are most likely due to the one experimental factor.

Health experts and policy makers have come to see the RCT as the “gold standard.” If an RCT shows some benefit for a drug, it must be good. If a psychosocial treatment, like diabetes self-management education, doesn’t have RCTs to document it, we shouldn’t use it or pay for it.

What’s wrong with the experimental model
Proving things with experiments is a good idea. The scientific method has overcome a lot of superstition and false beliefs. But experiments and science are misused in many ways and are used to spread all kinds of misinformation. Here are some reasons why scientific “evidence” doesn’t always hold up:

  • RCTs don’t work well when there is more than one intervention. If you’re studying the benefits of peer mentoring, as our blogger Eric wrote about (“P2P Diabetes”), it’s hard to randomize, since you can’t force someone to accept a peer mentor. It’s hard to control for everything, because not all peers will be the same. And it’s hard to say which one, particular piece of the mentoring helped. So your peer study will be ignored, if it can get done at all.
  • Bigger studies have more statistical power, so they get more attention. The more people you enroll in your study, the more impressive your results are likely to be. A relatively small benefit looks convincing if thousands of people experienced it. Drug companies have deep pockets to run huge studies. If Eric’s doctor’s lauched a peer mentoring study, it would be run on a shoestring, probably by staff volunteering after their other duties are done. It would have to be much smaller and therefore have less impressive statistical results.
  • Studies don’t consider individual variation—if one drug works for 40% of the people, and another for 30%, and placebo for 20%, the 40% drug will be recommend for everyone. In reality, the other drug, or no drug at all, may well work best for people like the ones who benefited from those treatments.
  • Scientists aren’t always honest or wise. They frequently misinterpret results. In the ACCORD trial, study directors concluded that normal blood glucose levels led to higher death rates in people with Type 2 diabetes. And media reported their conclusions as fact. These mistakes often seem to reflect the interests of those who paid for the study.

Cardiologists studied whether providing diabetes information to people with diabetes and heart disease would improve cardiac outcomes. They found no helpful effect. They publicized their results in The American Journal of Cardiology as indicating that all forms of diabetes education were a waste of time, and the money should be spent on statin drugs instead. But we already knew that diabetes information alone doesn’t help in most cases. It’s the skills and support for changing behavior that make the difference, as has been shown in many studies.

My take-home message is this: The scientific method is wonderful. But modern science is often a rigged game, not an honest search for truth. Pay attention to scientific reports, but take them with a grain of salt, and investigate for yourself.

What do you think? Please let us know by commenting here.

POST A COMMENT       
  

Comments
  1. Dear David. Yes you are right to question all of this. There was an British editor of a medical journal on Charlie Rose on PBS who said that studies have a remarquable tendency to prove the conclusion that the sponsor has paid for. As an Engineer I think that using RCT to determine the correct theory is funny. Imagine Isaac Newton using RCT to establish his theory of gravity, even more funny Einstein’s.The problem could be mainly “all other things being equal”. If the Fuhrer or the Japanese Imperial Army had won, very controlled experiments could have been carried out. But even then I am not sure that RCT works. Fundamental knowledge of how the body works may be the only way. The probe that landed on Titan go there because everything about the total system was reasonably understood. This may be a long way away in medical “science” so we have woodoo in the interim. If RCT contradicts common sense go for the common sense.

    Posted by CalgaryDiabetic |
  2. Dear David. There is another great problem in advancing medical “science”. One of my daughters after 15 years of schooling was working this summer in fundamental biological research and her pay was less that if she had been working at McDonald’s. This sent her a very negative message about the value that society places on medical knowledge. So she is becomming a Dentist who will most likely not contribute anything or very little to medical knowledge. Another very bright technical person I know is playing money with the big boys on Wall street.

    Posted by CalgaryDiabetic |
  3. You know what makes me sad? The FDA, the said government agency that suppose to be the gate keeper for us to prevent malicious drugs from entering the market is worst than having no agency. People trust the FDA and the FDA approved drugs (to their sorrow). If there were no FDA, at least everyone would be suspicious of new drugs before blindly accepting them.

    Posted by Bahamut |
  4. I really like this debate. The same problems arise in the behavioural treatment of autism. When the ideal RCT can’t be found, all the other evidence for using Applied Behaviour Analysis is ignored. Would you have any references that I could chase up on the misuse of RCTs in science?

    Mickey

    Posted by Mickey |
  5. Donald Berwick, director of the Institute for Healthcare Improvement, has spoken and written about the limits of RCTs. HEre’s a link to a brief view of his analysis. http://www.achse.org.au/documents/mythbusting-workshop-v2.pdf

    A good article on how RCTs don’t describe or analyze behavioral interventions is at http://www.musictherapyworld.de/modules/archive/stuff/papers/ClinRes.pdf

    I’m sure there is much more, not so much on “misuse” of RCTs, as their inapplicability to much of health care.
    David

    Posted by David Spero RN |

Post a Comment

Note: All comments are moderated and there may be a delay in the publication of your comment. Please be on-topic and appropriate. Do not disclose personal information. Be respectful of other posters. Only post information that is correct and true to your knowledge. When referencing information that is not based on personal experience, please provide links to your sources. All commenters are considered to be nonmedical professionals unless explicitly stated otherwise. Promotion of your own or someone else's business or competing site is not allowed: Sharing links to sites that are relevant to the topic at hand is permitted, but advertising is not. Once submitted, comments cannot be modified or deleted by their authors. Comments that don't follow the guidelines above may be deleted without warning. Such actions are at the sole discretion of DiabetesSelfManagement.com. Comments are moderated Monday through Friday by the editors of DiabetesSelfManagement.com. The moderators are employees of Madavor Media, LLC., and do not report any conflicts of interest. A privacy policy setting forth our policies regarding the collection, use, and disclosure of certain information relating to you and your use of this Web site can be found here. For more information, please read our Terms and Conditions.


General Diabetes & Health Issues
Getting to Sleep and Staying There (09/24/14)
How Much Do You Know About Diabetes? Six Facts to Get You Thinking (08/25/14)
Doing Your Own Research (08/06/14)
Ensuring a Successful Hospital Stay (08/15/14)

Diabetes Research
Can Grapefruit Juice Prevent Weight Gain, Increase Insulin Sensitivity? (10/16/14)
Low-Carb Diet Improves Quality of Life in Type 2 Diabetes (10/07/14)
Long Hours at Low-Income Jobs Linked to Increased Diabetes Risk (10/02/14)
Statins May Reduce Risk of Nerve Damage, Other Diabetes Complications (09/18/14)

 

 

Disclaimer of Medical Advice: You understand that the blog posts and comments to such blog posts (whether posted by us, our agents or bloggers, or by users) do not constitute medical advice or recommendation of any kind, and you should not rely on any information contained in such posts or comments to replace consultations with your qualified health care professionals to meet your individual needs. The opinions and other information contained in the blog posts and comments do not reflect the opinions or positions of the Site Proprietor.