Avoiding Internet pseudoscience

Thank you for joining our chat today. Please feel free to submit your questions. We will start at noon.

Thanks so much for joining us for this chat. The topic is so important in this modern day. We are totally bombarded with health information on the Internet and much of it is totally bogus. Dr. Robert Goldberg is an expert on this topic – he wrote an excellent book called Tabloid Medicine: How the Internet Is Being Used to Hijack Medical Science For Fear and Profit. Okay, welcome Robert, and welcome readers. Let’s go!

Dr. Goldberg, what do you think of the quality of information on the internet, when it comes to medical information?

A great question to start off the discussion. It depends on where you go for information. Scientific discussions require confirming evidence and providing a context…most information on the web does not include either.

On a scale to 1-10 I would give the information a 5 because the majority of stuff is so poor but a few sources are great

hi Todd, Randomized clinical trials are often considered the gold standard. In many cases they are if they are just looking at how one factor affects an outcome.
At the same time studies that capture lots of factors and follow them over time can be just as reliable. I think the worst type of research is the meta analysis because you can cherry pick the studies you want to get the outcome you want. And we are seeing more of these because they get headlines

The other really important part is to remember that studies can’t be considered alone – we need to think about whether they have been replicated by other independent teams of researchers. One study – even a well designed RCT in a respected journal by researchers at a prestigious institution – means less if it hasn’t been replicated.

Great question for two reasons. First as Trine notes, you have to look at the context and whether a study has been replicated. Second, as the slew of studies claiming diet drinks, sitting around or limited exercise are linked to heart attacks ‘show’, failing to look at other health factors skews the conclusion. I am less concerned about who is distilling than whether researchers are doing studies that can be replicated and take into account a variety of factors including biological risk

Hi Anne. That is so true. It is very difficult for a lay person to determine whether a study is good, or what it means. Even specialists in one field may have difficulty really understanding a study in another field. So we depend on people to interpret it for us. Ideally, you want the true experts distilling it for you. Dr. Vincent Racaniello at virology.ws is a good example. A respected virologist with a knack for explaining complex topics to laypeople.

I would avoid Gary Schweitzer… he is a amateur (read, not a scientist) who seeks to undermine the credibility of studies regarding the value of medical innovation without applying Trine’s rules. Opinion is more subjective than science. 🙂

In my reporting on alternative treatments for autism, I was always struck by the wide chasm between what folks peddling dubious treatments said certain papers meant, and what the researchers who actually wrote the papers said they meant. There was often a massive difference. And yet, for a regular person, like me or most folks, that would be hard to determine.

Well, certainly some journals are better than others. Medical Hypotheses is not a place I look for solid medical studies. Science is. JAMA is. That said, studies in the most respected journals can be wrong, and often are. John Ioannides at Tufts has studied this phenomenon, looking at old studies in respected journals and seeing how many turned out to be right. The percentage was strikingly low. So you can’t trust studies in isolation in good journals. You have to be skeptical, and think about what the body of work on that question points to.

There are certainly some journals that are regarded as “leading” because lots of people follow them or because the media deems them leading. But remember that it was the British Medical Journal — a ‘leading” medical journal — the published Wakefield’s fraudulent connection between vaccines and autism.

That is a great point. Wakefield’s terrible study was published in a very respectable journal. And it was just terrible.

And I would also refer to a great article in Commentarymagazine.com, “The Worst Study Ever” that discusses the biased approach to rating health care systems carried out by the World health Organization, the one that claims the US is 37th in overall health.

I think it is worth remembering always that scientific knowledge is hard won – it is gathered so slowly. One study adds to what we know about a particular question, but one study rarely answers that question alone. So whenever you see a study, it’s worth looking for others and to keep looking and reading. Ask yourself, what, in total, do we really know about this?

Yes.. and they can all be found in my new book (of course!).. Seriously, my rules of the info superhighway are:

1. If someone is selling something and claiming science supports it, don’t buy either.
2. A single study with a small sample is to be avoided
3. If the study claims an association instead of biological cause and effect, it is less reliable, especially if the cause and effect is not replicated by other studies

I love it when a site displays that warning and then goes on to make claims about how their product IS intended to diagnose, treat, cure and/or prevent disease, often a lot of them. That’s always great.

I think it does have an educational effect.. but hope is a dangerous thing. Because most of the products that require that label claim they are ‘natural, safe and effective’ in the absence of any other information to the contrary people will buy alternative products.

I do think it’s useful in that it makes it clear that the products are not FDA-approved.

I would also add that there is a booming industry — funded by government — for producing studies that claim a cheaper treatment is better than a newer one. It’s called comparative effectiveness research. Yet there is no research showing whether such older treatments are actually more effective. It’s merely a literature review with an opinion. These studes ALSO have a quack miranda warning by the way..

A great question, and a timely one. There have been recent scandals about conflicts of interest not being disclosed in studies published in big respectable medical journals.

It has almost no difference. First, to claim that funding affects conclusions would be to suggest that every discovery that has lead to vaccines, chemotherapy, heart treatments supported by “industry” really don’t work. Was Pasteur’s fermentation research a fraud because it was funded by the French beer and wine industry.

Second, funding is different that conflicts. Most of the most unreliable — and deadly — research came from people that had an ideological or cultural bias. Wakefield is the best example of that.

Oh, what a terrific topic. Right now, in the world of CFS, there are several non-scientists CFS patients who are attempting to “school” career retrovirologists on how to do their experiments, on what is wrong with their technique. Their PhDs are from Google U, of course. What hubris. This is becoming more common. It’s great to learn about another field, but no amount of studying at Google U turns a layperson into a real expert.

Great point. It’s the reason that people have stopped immunizing their kids, stopped getting treated for depression, etc. And sadly a whole new body of research claiming certain treatments are not only ineffective but dangerous as well based on cherry picking of studies are easy to digest and spread on the net. In the absence of no knowlege, information warning danger is likely to be accepted.

I was at a conference a few years ago and someone introduced me to the “world expert on vaccine studies.” She was described as such to me several times. I was excited to meet this person. I met her and it turned out she was a Spanish teacher with no training in epidemiology at all. But others considered her an expert because, well, who knows why? But she had managed to spread a lot of misinformation about vaccines and autism. She had done harm because of this Google U phenomenon.

This is a challenge I face every day as a journalist. I know I know so little about extremely complex fields, and yet I need to figure them out. A real challenge for anybody delving into something new in medicine.

And when I say “figure them out,” I don’t mean I actually become an expert. Not even close. But I need to get my arms around what is going on, as best as I can. And that is difficult.

Confirmation bias is hard to avoid and even worse, researchers and advocates use studies to sway opinion and behavior based on a desire to avoid risk. But science is the collective process of testing hypotheses and holding experience up to scrutiny. And it is – or should be — increasingly a process of establishing what are the factors and biological mechanisms that cause disease. We are healthier and live longer because of that collective enterprise. I am afraid the internet fragments and does an end around that process.

Oh it does! A good example right now is the XMRV/CFS debate.

“Unpublished data” usually means unfit to present to the rest of the world because the sample size is small, the design is experimental or the conclusions are so significant that they really have to be a statistical fluke… All the ‘unpublished data’ claiming a link between suicidal behavior and antidepressants AND that antidepressants are no better than placebo (so a drug no better than placebo is actually powerful?) was stuff that was really unreliable. But it did become a weapon in the hands of people who had an axe to grind.

This is so true. Big Pharma and Big Government are often cast as villains by dubious medicine practitioners. And of course, there have been scandals involving Big Pharma and Big Govt, which gives even the wildest conspiracies fuel.

Great observation. I happen to think you are right, that much of the tabloid medicine movement is part of a larger effort to tear down institutions and defend the “oppressed.” Aaron Wildvasky, a political scientist noted that there is nothing more effective in attacking science or technology than claiming it causes cancer. He was speaking about environmentalism but it also applies to efforts to attack medical technology.

No better example of this is conspiracy described by some who say vaccines cause autism. There has be a way to explain away the studies, which found no evidence of a link, right?

So many folks selling so many treatments to parents with children with autism, based on junk science, with no proven benefits and in some cases, known risks. In others, unknown risks. And yet, docs selling these dubious treatments often accompany their sales pitches for them with lots of scientific papers. Trying to sort them out is an enormous challenge. A full-time job, really. But when you do, the whole of it winds up to be junk. We wrote about this in a series that won national awards: bancodeprofissionais.com/autism

More than that (and great last name by the way) the so-called medical experts the media ran to for comment also validated it by not speaking plainly about the lousy quality of the data. I would add that when someone like Steve Nissen, a cardiologist at the Cleveland Clinic, claimed a diabetes drug was “another 9-11” he was doing as much damage as Wakefield did. Fewer people are taking oral diabetes drugs and the bar for new drugs is higher than ever.

And then you have folks like Dr. Mehmet Oz, who has had a tremendous career as a cardiac surgeon. And yet now he is selling his audience on faith healers and folks who claim they can talk to the dead. He has a website that is jumbled with evidence-based information and stuff that is absolutely not based in science. How to sort it out? Is he for evidence-based medicine or not? I would say it seems not. But yet he is out there as a credible source of info.

I think it is not a failing of the medical community that studies often turn out to be wrong. It’s the way science works, it is self-correcting over time. Scientists know that. It’s the media who get it wrong and tout one study in isolation as an answer to a medical question when it never is. Never. We should not write about single studies that say, oh, coffee will give you cancer. Because tomorrow, it is entirely possible another study will say it is protective. Over time we will figure it out. It doesn’t mean scientists are stupid and can never get it right. it means science is hard, and that we need to be patient and wait for knowledge to develop over time.

You mean” Why Most Published Research Findings Are False.” That paper makes two important conclusions: combining lots of small, unreliable studies will produce a very big literature view that is hugely unreliable. Second, that bias is not just financial, it is intellectual, political, cultural and personal. Most scientific fraud is committed by people who want to advance their careers. And much good research is ignored because it does not fit the status quo. The paper is a great reminder about how the collective intellect of science — over time and place — is the only sure path to progress.

A great example, not to tout XMRV-CFS again, is that patients with CFS are taking HIV antiretroviral drugs right now based on one study published in Science that associated CFS with the retrovirus XMRV. Many other teams have failed to replicate that original study, and yet, there are patients looking at that one study, deciding it has to be right, and actually taking potent drugs with serious side effects because of it.

Hi Kris. I think that is a great strategy.

The internet is a tool, not a magic wand. It is being used by people using pseudo-science to push an agenda to a greater extent than scientists who could weigh in with useful and more scientific information.. That’s the real problem. The go-to people are few and far between and they are often ignored because of the narrative that financial ties to industry means the experts are corrupt or unreliable. We need more courageous voices like Paul Offit

Yes, I agree. I saw a comment in an article about online health that underscores your concern: “The patients are already on the Web.. they are waiting for scientists and doctors to join them.” The web can be a great tool for holding the media and pseudoscientists to account.

I can’t talk about any other reporters’ expeirences, but in my experience, scientists have been happy to talk to me. They return my calls. They email back. They are happy to fact-check and they have been patient about explaining often technical information to me. I do think it is a massive challenge for a scientist to explain his enormously complicated field to a layperson like a journalist. Some have been burned before, misquoted, etc. So perhaps occasionally they simplify too much. But my experience has been very good. It’s our fault in the media for getting it wrong, I think.

Desperation is a powerful emotion. And now we have so many places – on the Internet – designed, really, to attract desperate people. Promising cures and hope based on junk science. And for a price.

Unfortunately it is not always possible for doctors to spend as much time as they would like.. and increasingly health insurers are trying to “script” doctors to lay out options in ways that often put the risks of treatment first. It is incumbent upon more doctors and scientists to reach out via the Web and good reporters like Trine to make sure that when people use the Web, the doctors are already there.

Gosh thank you so much Robert!! You are so sweet. Well, sadly, I think we have run out of time, and what a great chat! Thank you so much for all of your fantastic questions. And thank you especially to Robert, who took time out from his trip to Israel to be with us! Join us next week for a chat with my talented colleague, Deb Shelton: April 26 at noon CT about children’s developmental skills.