Have you heard about the latest medical breakthrough? It’s a miracle cure that reduces the risk of brain cancer by 52.737%! Brain Shield was recently unveiled at the Meeting of those Concerned with Brain Cancer with stunning success. But don’t take my word for it. Annie from Lodi says “What a godsend. This treatment changed my life.” Annie has been using Brain Shield for 13 months and guess what, no brain cancer. Brain (CT) scans have confirmed that Annie is cancer-free. Interested in reducing your risk of cancer? Call 1-878-4-SHIELD.
Sound suspicious? Well, it should, because it’s completely bogus. I’m certain most of you weren’t fooled. But, did you pick up on the specific clues that Brain Shield story is full of bull#$%*? There are a number of them. By discussing each red flag, I hope to provide some tips for cutting through the hype in the medical media. Your health is of the ultimate importance and the quality of the medical news you heed (or ignore) is critical. So, here’s how to discern the valid from the bull #$&*. I thank the Association of Health Care Journalists (AHCJ) for providing the basic principles to work from.
Red Flag #1: Language. Be wary of health reporting that utilizes sensationalistic language. I am not sure that “miracle cures” actually exist, but if they do, they are rare. Antibiotics might qualify. A (hypothetical) treatment to reduce the risk of brain cancer by 53% certainly does not. The vast majority of “breakthrough” treatments and “dramatic” findings are nothing of the sort. Similarly, phrases like “deadly diseases” and “sweeping epidemics” are used to sell media, not to properly inform.
Red Flag #2: The anecdote. Testimonials may be useful in painting a vivid and personal picture of a disease or treatment but anyone who extrapolates the experiences of a handful of people to larger groups is taking a leap of faith. The AHCJ cautions health journalists to avoid the “tyranny of the anecdote,” particularly when the anecdote contradicts the sum of available evidence. An excellent example is the long hyped (and many times disproven) link between vaccines and autism. Causality is difficult to establish and a series of heart-breaking stories about children who developed autism not long after receiving immunizations does not establish a link and absolutely contradicts the vast evidence exonerating vaccines as a causal agent. Similarly, absent more information, readers should not be impressed by the “remarkable story” of Annie from Lodi.
Red Flag #3: Lack of peer review. The peer-review process in the scientific community is far from perfect, and plenty of junk slips through its cracks and into the medical literature. Andrew Wakefield’s seminal publication about the MMR vaccine and autism has become a classic example. But nonetheless, peer-review is a rigorous process – and one that (mostly) discourages researchers from publishing poorly conceived or managed research. Thus, you should put far more trust in medical evidence presented in a peer-reviewed journal (such as the New England Journal of Medicine) than preliminary results that appear in the mass media or at a scientific conference. Research presentations at conferences have not been vetted in the same manner as studies that make it to print in journals. The Meeting of those Concerned with Brain Cancer sounds like a worthwhile gathering, but should not be taken as the definitive source for information about preventing brain tumors.
Red Flag #4: Lack of perspective. There is a lot of nuance involved in interpreting the numerical significance of medical research. A 52.737% reduction in the risk of brain cancer sounds fantastic until you consider that brain cancer is an extremely rare condition (about 6 cases per 100,000 people). Think about it in terms of the value of your home – a 53% reduction in value is profound if the starting value is high. But what if your house isn’t worth squat (let’s say $100) to begin with? Then a 53% difference is no longer a big deal. Similarly, a 53% reduction in the number of cases of a common medical condition is big news, but a 53% reduction in cases of a rare condition is less newsworthy. Journalists can put this into perspective by giving data about statistical significance (the benchmark as to whether an observation is likely to have occurred randomly or is likely related to an exposure) or – for medical therapies – the number-needed-to-treat. For example, our hypothetical Brain Shield would need to be used in nearly 200,000 people in order to prevent a single brain cancer. When you consider this, you also realize the use of extra decimal points (52.737%) is another red flag. Those extra decimal points are purely for show – a means of making the results seem more robust than they actually are. Genuine data does not require three decimal places – none or one is usually sufficient. Watch out for numbers with drawn out decimals.
Red Flag #5: Too good to be true. No medical treatment is without risk. This is even true of treatments involving basic life substrates such as oxygen, water and salt. Too much of any of these things can be harmful. Thus, any discussion of a medical treatment must mention its risks. The very rough estimate of lifetime risk of cancer caused by a CT scan of the head is one in 2000 – thus Annie from Lodi has actually increased her brain cancer risk by attempting to confirm that her anti-brain cancer treatment is working. Silly Annie.
So, next time you read “stunning” health headlines or stay up for the 11 o’clock news to learn about the latest epidemic sweeping the area, keep these red flags in mind. If you encounter any, tune out, go to bed, and find a different (better) source in the morning. Or if you’re not sure, you can follow the advice of my buddy (and frequent contributor) Dr. Clark Hinderleider and “investigate the reliability of the outlet from which the content is received by using a 'fact-check' source such as HealthNewsReview.org." Whatever you do, I wouldn’t rely on Annie’s advice.
