Overcoming Probability Inflation

Benjamin Roman, MD, MSPH wrote a wonderful piece in this week’s New England Journal of Medicine. It might not get read much because it is listed way down the table of contents but I think it is more clinically important than any other piece in the journal this week. He tells of his own story of having sudden sensorineural hearing loss and agreeing to an MRI even though the probability of him having a serious cause of the problem was low, the cost of the test (MRI) was high, and the benefit of treatment was minimal (in fact, many don’t need treatment). Furthermore, he is an ENT physician and knows all this but still underwent testing anyway- mainly because his wife wanted him to!

He outlines an important problem in medicine for both physicians and patients: probability inflation.

This problem arises from the way we deal emotionally (added for emphasis) with risk and uncertainty, which are givens in health care, and the way we make decisions in the face of low-probability outcomes.

Emotions are a large part of the problem; the affect heuristic. When we make decisions we often consider it analytically but also from the standpoint of how we feel about it. If we have positive feelings about the situation we magnify the probability of benefit or, conversely, reduce the magnitude of harm. Think about Dr. Roman’s situation. He (or at least his wife was) was worried about something bad happening (ie having an acoustic neuroma) but understood that was pretty unlikely to be the case. But what if he didn’t do the MRI and he actually had a treatable one that would be missed. He had strong feelings (or at least his wife did) that he didn’t want to miss the acoustic neuroma. Or maybe he would be relieved that he didn’t find one (that’s is a strong positive emotion isn’t it) if the MRI was negative (assuming the sensitivity is good enough). Thus, the acoustic neuroma’s probability becomes artificially inflated. He probably didn’t even think about the downstream effects of finding one and the risks associated with having surgery or radiation (which probably outweigh the benefits of finding it if I had to guess).

Many of us fear the uncertainty almost more than the disease itself. We want to know even if we can’t act on the information we are given. We also like doing something. At least we will go down fighting. This affects both physicians and patients. We order things we shouldn’t. Patients request things they shouldn’t. Sometimes its because of poor reasoning skills. The affect heuristic gets us. Sometimes its more practical as Dr. Roman notes:

My doctor’s recommendation was based on a similar reaction. Besides wanting to reassure himself and his patients that there is no acoustic neuroma, he told me, another reason he suggests MRIs in situations like mine is that he fears being sued should he fail to order one and end up missing something. He noted that court malpractice awards for missed acoustic neuromas commonly reach into the millions of dollars and that until we agree to an acceptable miss rate and physicians are no longer liable for missing just a single such case, their practices will not change. I’m not sure how common such verdicts are, but this rationale also reflects risk aversion in the face of a low-probability bad event — it’s simply the doctor’s risk that’s at issue, rather than the patient’s. (emphasis added)”

That last statement is telling. It’s a shame so much of medicine revolves around covering our proverbial asses.

Dr. Roman offers some solutions:

  1. comparative effectiveness and outcomes research (this exists for many things but gets ignored)
  2. educating doctors about how to discuss uncertainty, risk, and probability (First, doctors need to be taught these principles before they can teach anyone else. I see first hand on a daily basis how little of this is understood)
  3. addressing emotions and psychology of patients and physicians (good luck dealing with emotions….. anyone have a teenage child?)
  4. nudging each other to do the right thing
    • consumers share cost of things they want that are marginal (good idea for sure)
    • government (either local or national) regulation (Hell no! More bureaucracy is not needed and will only raise costs even more)

As Dr. Roman points out all of these need to be done but the devil is in the details. HOW? I think the focus of these solutions is from a society or community perspective and physicians mainly feel a duty to only one individual- the individual sitting in front of them. That relationship is powerful and affects decision making.

My dad had advanced dementia and fell in his bathroom suffering a tibial plateau fracture. The surgeon wanted to fix it surgically as this would give my dad the best chance to walk (though he couldn’t actually tell me the probability). The only other option was splinting and rehab.  Thankfully, I know enough about dementia and specifically my dad’s dementia to know he would never be able to participate in rehab and I knew he would never be able to keep the wound clean and stay off his leg until it healed. I decided not to do the surgery and opted for rehab and splinting. My dad never walked again. He couldn’t understand how to do rehab or to use a walker. I made the right decision because I think the ultimate outcome would have been the same either way- not walking. I have no way of knowing. It was a decision under uncertainty. I saved his insurance and Medicare a lot of money. That wasn’t my goal. My goal was to maximize outcomes in the most resource-sensitive way that would harm my dad the least. I felt surgery would be more harmful than not doing the surgery. Should the surgeon have even offered to do surgery? Should he have just said that splinting was the best for someone like my dad with advanced dementia? When he offered surgery did he really thing it would help or was it because he was a surgeon and that’s what they do?

Like all complex problems the solutions are equally if not more complex. I will continue to do my small part of educating who I can on EBM principles and hopefully a few of my learners will make good decisions.


Affect Heuristic,COI, or Lack of Knowledge? Why Do Cardiologists Overestimate Benefits of PCI in Stable Angina?

A recent study in JAMA Internal Medicine by Goff and colleagues made me wonder if the Cardiologists studied are uninformed of the limited benefits of stenting (PCI) for chronic stable angina, do they have too strong of a conflict of interest due to economic gain, or is the affect heuristic playing a big part? Probably  a mixture of them all. The COURAGE Trial taught us that PCI was better than medical therapy at reducing anginal symptoms but wasn’t any better for reducing MI and death.

Goff and colleagues reviewed 40 recordings of actual encounters of Cardiologists with patients being considered for cardiac catheterization and PCI. I am unsure if these were video recordings or audio recordings. As best I can tell these were all private practice Cardiologists.  Cardiologists either implicitly or explicitly overstated the benefits of angiography and PCI. They presented medical therapy as being inferior to angiography and PCI (a statement that defies the findings of the COURAGE trial). In fact, in only 2 of the encounters did they state PCI would not reduce the risk of death or MI. These Cardiologists also didn’t use good communications styles that encouraged patient participation in the decision making process.

Why might these Cardiologists do this? They could be uninformed of the limited benefits of PCI in stable angina, but I doubt it. COURAGE was a landmark publication in one of the world’s most prominent medical journals. I find it hard to believe that Cardiologists wouldn’t be aware of the results of this trial.

They certainly have a financial stake in their recommendations.  The image below shows that a diagnostic cath is reimbursed at approximately $9,000 while a PCI with DES is reimbursed at approximately $15,000. That has to have an impact on decision making. I don’t accuse these Cardiologists of doing a procedure only for money but subconsciously this is playing a role. Recommending medical therapy only gets you an office visit reimbursement (maybe $200 or so).

What about the affect heuristic? My colleague Bob Centor writes about this often in his blog. A heuristic is a quick little rule we use to make decisions. The affect heuristic is a particular rule we use that is based on our emotions about a topic. Do I like it? Do I hate it? How strongly do I feel about it? The affect heuristic leads to the answer to an easy question (How do I feel about something?) serving as the answer to a much harder question (What do I think about something?) Its not hard to imagine (and data in the Goff paper supports this) a Cardiologist feeling that PCI is beneficial and should be done. They are emotionally tied to angiography and PCI….they have seen patients “saved” because of this procedure.

So what can be done? The solution is harder than determining the problem (as is often the case). The easiest solution is for insurance companies to stop reimbursing for the procedure in stable angina unless patients have failed optimal medical therapy but this is draconian. I also worry that patients will then receive bills for unreimbursed  catheterization charges. I think using the technology that was used in this study combined with feedback could be useful but logistically impossible. I have always wondered why we don’t use secret shopper fake patients to evaluate physician skills and knowledge (of course the answer is a logistic one) instead of the MOC system. Just publishing a study doesn’t work if physicians don’t read or if they don’t use that study to answer a clinical question. Patient decision aids (like this excellent example) could be very useful but the physician would have to use the tool and many don’t even know they exist.

Some would argue EBM has failed again. A well done study was published and it hasn’t made a difference. The principles of EBM have not failed and in fact, if they were used, could limit the inappropriate use of PCI in stable angina patients. What has failed is the desire to learn and use these skills by the older Cardiologists in this study. Like many physicians, they rely on outdated knowledge and emotions or beliefs. As stated by Bob Centor in a post about the affect heuristicDecision making bodies have biases. Until they understand their biases, we will have the problem of unfortunate, unnecessary and potential dangerous unintended consequences“. In this case the Cardiologists are the decision making bodies and the unintended consequences are the MIs, strokes, renal injury, and death that can and do occur from cardiac catheterization.

Hopefully you are now aware of what the affect heuristic is and how it impacts decision making. Acknowledge it and separate your feelings about a topic from the data. Your patients will benefit.