Tag Archives: EBM

Standard

Posted by

Posted on

August 19, 2016

Posted under

General EBM

Comments

2 Comments

Tarnished Gold Chapter 3: Prove It

This chapter dealt with the issues of what constitutes evidence. Instead of focusing on their views I will focus on my views of evidence.

the_evidence_tv_series

A common criticism of EBM is that it very strict in what it considers  acceptable evidence and it doesn’t consider clinical experience and pathophysiological rationale as important. Early EBM did focus too much on the RCT and Cochrane systematic reviews but this has changed. The current EBM paradigm focuses on multifactorial “evidence” including the patient’s clinical state and circumstances, clinical experience, and the best available evidence. Sometimes this will be a systematic review but often it will just be patient experience (what worked or didn’t work for them in the past) or pathophysiology. The early EBM paradigm cautioned us that we can be misled by our unsystematic observations and the pathophysiological rationale. For the latter, it’s because our understanding of pathophysiology changes and diseases are complex and multifactorial and interventions we study tend to be unifactorial. Nonetheless, clinical experience is evidence and is very important and no EBMer will say otherwise. Understanding pathophysiology is important and no EBMer will say otherwise. The key is to understand the limitations of any evidence source.

Evidence supports a belief and doesn’t have to be true. In clinical medicine we can never know the truth. We can only try to estimate the truth with a study because we can’t study every person with a given disease. We have to infer a lot. We generalize from a sample in a study to a whole population and back down to an individual patient. The authors of Tarnished Gold have a real problem with this paradigm but it’s what we do in clinical medicine. Bench research works differently. Rats can all be genetically and phenotypically the same. Bacteria can all be clones of each other. Bench scientists can study a whole population of something and declare an effect. We can’t do this in clinical medicine because we are all so heterogeneous and have free will.

EBM no longer worships only the RCT and the Cochrane review. Patient inputs are viewed as very important and slowly becoming equally important. Qualitative studies are gaining importance. Clinical experience will always be prominent in deciding what should be done from what could be done.

Standard

Posted by

Posted on

August 16, 2016

Posted under

General EBM

Comments

Leave a comment

Tarnished Gold Chapter 2: Populations are not people

Populations are not people

First off the authors state that decisions sciences do not relate to EBM. They feel decisions are personal and statistical information is not important. They give the example of organ transplantation. Unfortunately, they skip an important step in their argument. Namely, that to know an organ transplant will be of benefit is based upon studies proving that they prolong life and these are based on statistical information.

They argue that EBM is based on a statistical blunder: the ecological fallacy.  There is some merit to this argument. The average finding applies to the average patient. What if your patient isn’t average. There are a couple of options. First, you could calculate your patient’s estimate of benefit (or risk) using the results from the study like I demonstrate in this video.  Almost every study report will include a confidence interval around the point estimate of benefit (or harm). The point estimate is the best guess about the findings of the study but there is uncertainty and the confidence interval helps quantify that uncertainty. You could use the upper and lower bounds of the confidence interval and decide if it includes a clinically important benefit. Finally, you could look for a subgroup analysis (yes I recognize the limitations of this) of a group of patients similar to yours. Despite all this, science is based on inference. We can never measure the effect of an intervention in all people. We often use inductive and deductive reasoning in science.

The authors spent several pages discussing pattern recognition in medicine and that EBM doesn’t help this. This is both true and false. It is true in that we are taught how certain things look and there will never be a study related to that. We have numerous studies though of how good elements of the history and PE are for diagnosing disease. Many of these are pattern recognition. We learn that peripheral edema, orthopnea, PND, and DOE are most likely congestive heart failure. That is pattern recognition but there is also a study that examines how good each of these components is to increasing or decreasing the probability of CHF. Thus, pattern recognition is informed by EBM.

There are more claims to be refuted in this chapter but these are the main ones worth refuting.

 

Standard

Posted by

Posted on

August 10, 2016

Posted under

General EBM

Comments

1 Comment

Tarnished Gold Chapter 1: Evidence-based Medicine

This is going to be a lot harder than I thought. I question why I am even wasting my time reading this tripe but I will plod forward so that there is a counterargument to this work. I also need to understand criticisms of this paradigm so that the paradigm can be improved.

Importantly, the authors focus on an outdated definition of EBM. This definition was the first iteration of the definition and is oft-quoted but it is out of date nonetheless.

Evidence based medicine (EBM) is the conscientious, explicit, judicious use of the current, best evidence in making decisions about the care of individual patients.

The current paradigm of EBM was published in 2002 well before this book was published and should have been included in this book. Hickey and Roberts claim it had its origins in the legal system which is total BS. If you read the early EBM papers there is no mention of the legal system driving this paradigm.  I also consider EBM to just be a set of skills (searching, critical appraisal, application) to use in the care of patients.

 

 

They focus and have problems with 2 words in the above definition: best and evidence. They are concerned that best leads to selection of evidence and that “one bit of evidence is better than another”. Of course some evidence is better than others. Empirical studies (not done by the evil drug companies) have demonstrated that certain design flaws, for example lack of blinding, lead to overestimation of effects. Studies have also demonstrated that observational study designs can overestimate effects and even give opposite effects to randomized trials (see the HRT saga). I’m sure they will argue later in the book that all these types of studies are rigged and randomized trials are rigged (probably because their holy grail, Vitamin C, failed in controlled trials to be useful). There are too many studies showing similar effects to discount the evidence that supports the fact that some studies are better than others.

They claim “EBM’s evidence does not mean scientific information or data, but refers to legal justification“. First off, EBM does not possess evidence so the ‘ is misplaced. Second, this statement doesn’t even possess any face validity. Journals are full of scientific information and data. What are they talking about?

They claim “EBM has little to offer the doctor treating a patient, beyond suggestions about what might be expected with an average patient“. Studies used to inform practice usually are based on a sampling of patients because we can’t study every single person with that problem. Sampling can be done to reflect a broad range of people with a given problem or it can be done to select for certain subpopulations of disease (for example, advanced disease or early disease). On average, most people are average. So their statement isn’t totally without merit. We can’t do studies on every type of patient. But, here is where the current paradigm helps us. We (as doctors) take into account the patient’s state and circumstances when applying the best available evidence to their case. We use our clinical training and experience to decide what we should do from what we could do. There are ways to adapt study data to an individual patient like I demonstrate in this video. N-of-1 trials can also be done on individual patients to see if a therapy is effective (more on this in another post).

Finally, (though there is a lot more I could comment on) they have problems with using statistics to analyze data. As I mention above, in medicine we can only sample a small percentage of those with disease. If we could study everyone we wouldn’t need statistics but since we can’t we use statistics on a sample to try to generalize it to the whole population. I don’t know of any other way to do this because we simply can’t study everyone. (I recognize this is a gross simplification of what statistics do and not totally accurate.)

The next chapter I’ll critique is entitled “Populations are not People”. Stay tuned…

 

 

 

Standard

Posted by

Posted on

August 10, 2016

Posted under

General information

Comments

2 Comments

Rebuttal to Tarnished Gold: The Sickness of Evidence-Based Medicine by Hickey and Roberts

Unfortunately, a lot has been going on in life and I haven’t posted as much as I would like to.   The main reason has been some major life changes and class work (I’m getting a master in educational technology). Another reason has been a lack of inspiration. Well, I am inspired now.

Tarnished Gold book cover image

Recently on a listserv I subscribe to someone mentioned a book called Tarnished Gold: The Sickness of Evidence-based Medicine. Some said it was a great book and many said it was garbage. I had never heard of it but could tell by the comments it is an anti-EBM piece. The EBM paradigm is evolving, as it should. Part of how a paradigm evolves is critical review of its tenants.  This book seems to challenge the tenants of EBM.  It’s important to look at all sides of a story to formulate a complete understanding of it. My bias is for EBM principles. I plan to read this book and give my rebuttal to each chapter. I will be as fair as I can and give them credit for good ideas and challenge ones I don’t agree with.  I have read the preface, foreword, and 1st chapter and know I will have a lot to write about.

Look for my 1st installment tomorrow.

Standard

Posted by

Posted on

January 8, 2015

Posted under

General EBM

Comments

Leave a comment

What is EBM?

With all the discussion of EBM in crisis and EBM on trial it strikes me that maybe these other folks have a different definition or concept of EBM than I do. I think to have any discussion needs to come from a common ground of just what is EBM.

Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.

This is the original definition of EBM published in 1996. It urged us to strive to use best available evidence in making clinical decisions. It also cautioned not to be a slave to evidence as evidence was often not applicable to individual patients. This definition served us well until the patient-centered paradigm of care became popular and the definition of EBM evolved to its current form:

This definition is more explicit about the order of importance of the individual elements of the components of EBM: patient preferences and actions is foremost, followed by the clinical state and circumstances and the research evidence. All this is tempered or tied together by our clinical expertise. The evidence tells us what could be done while the rest tells us what should be done.

The other way to look at EBM is that it is just a set of skills:

  1. asking an answerable clinical question
  2. finding the best available evidence
  3. critically appraising the evidence
  4. applying the evidence to individual patients
  5. appraising how well you did on each step and, I think, appraising the impact on a patient

So from this background I find it difficult to lay blame on EBM for many of the problems with the evidence. I blogged on this previously and will refute their claims at EvidenceLive2015 in April.

Standard

Posted by

Posted on

January 4, 2015

Posted under

General EBM

Comments

1 Comment

Will EBM be found guilty or not guilty?

Carl Heneghan recently wrote a blog for BMJ blogs entitled Evidence based medicine on trial focusing mostly on the problem with the evidence part of EBM. While I mostly concur with his list  of the problems (distortion of the research agenda, very poor quality research, and lack of transparency for published evidence) I wonder who is at fault. “EBM” seems to get the blame as if there is an entity that is EBM and it controls all research. EBM is but a set of skills: question asking, searching, critical appraisal, and application to individual patients. It is nothing more. So why are people being so critical and place so much blame on a set of skills? There will be several sessions at EvidenceLive 2015 (one of which I will be speaking at in defense of EBM) on real vs rubbish EBM.

I want to focus on the distortion of the research agenda. Professor Heneghan rightly points out that the research agenda is driven by industry. Is that good or bad? I think its both but mostly good. The only other major funders of research  are governmental agencies like the NIH. Profit drives innovation. It is very expensive to bring a drug to market. The government could not afford to bring the current drugs we have and need to market. One failed drug alone would deplete the coffers. Failure is the biggest driver of cost. Fewer than 1 in 10 drugs tested makes it to market. Would we tolerate that poor of a success rate at such a big cost by the government? No.

…adjusting that estimate for current failure rates results in an estimate of $4 billion in research dollars spent for every drug that is approved.

I agree that industry seems at times to make a drug then find a “disease” for it. I think the example Professor Heneghan gives is spot on. I don’t believe in adult ADHD but we have drugs for it. Do we need them? No and this video demonstrates why:  Drug free treatment of ADHD. Who is really at fault are the doctors who prescribe the drugs that Professor Heneghan feels aren’t necessary. Not the companies for making them.

On a serious note…what about all the devices we use regularly like stents, defibrillators, etc? Would government have independently brought these to market? Likely not. We had balloon angioplasty (without stenting) that worked just fine albeit short term only. It would have been “good enough for government work” as the saying goes. What about advancements in imaging modalities? Again likely not. The old CT scanners worked just fine. Industry is largely responsible for innovation and improvement in all walks of life. Yes for a profit but profit is not a bad thing. Those who say otherwise please return your iPhones.

Standard

Posted by

Posted on

August 1, 2014

Posted under

evidence based practice, General EBM

Comments

Leave a comment

Evidence Based Medicine Is Not In Crisis! Part 4

I’ve left the hardest issue to deal with for last- “Overemphasis on following algorithmic rules”.  This has been the most frustrating aspect of my primary care practice.  Patients quit being viewed as patients but a set of goals that I had to achieve to be smiled upon fondly by my boss as being “a good doctor”. It took me some time to finally quit playing the game and just do the best I could do and whatever the numbers were so be it.

Algorithmic medicine couldn’t be any more antithetical to EBM. Everyone is viewed the same. EBM clearly, as I have argued in the last three posts, is about individual patient values and circumstances. It’s about clinical experience temporizing what we could do to what we should do.  Algorithmic medicine allows no individuality.  No temporizing. Thus to claim EBM is in crisis because of algorithmic medicine is wrong. True EBM protects us from the harms of algorithmic medicine.

Interestingly computerized decision support systems (mentioned as a culprit in the first sentence of this section of Greenhalgh’s paper) are at the top of Haynes’ 6S hierarchy of preappraised evidence.

“In these computerized decision support systems (CDSSs), detailed individual patient data are entered into a computer program and matched to programs or algorithms in a computerized knowledge base, resulting in the generation of patient-specific assessments or recommendations for clinicians” –  Brian Haynes

At the VA we have a moderately sophisticated CDSS. It warns me if my patient with heart failure is not taking an ACE inhibitor and its smart enough that if I enter an allergy to ACE inhibitors it won’t prompt me to order one. If I tell it that a patient has limited life expectancy it will not prompt me to pursue certain routine health screenings. Thus, I don’t view CDSSs as problematic in and of themselves. The problem arises when physicians don’t consider the whole patient (remember those values and clinical circumstances) in deciding whether or not to follow prompted recommendations.

Greenhalgh has made great points about what happens when good ideas are hijacked and distorted for secondary gain but EBM is not to blame. Victor Montori (@VMontori) said it best in a Tweet to me:

“EBM principles are not in crisis, but corruption of healthcare has oft hidden behind the e-b moniker. EBM helps uncover it“.

 

Standard

Posted by

Posted on

July 28, 2014

Posted under

evidence based practice, General EBM

Comments

3 Comments

Evidence Based Medicine Is Not In Crisis! Part 3

In this installation I want to jump ahead in Greenhalgh’s paper to address her last cause of the EBM crisis: “Poor fit for multimorbidity“. Not to worry, I will come back in a future post to cover the remaining “problems” of EBM.

I concur with Greenhalgh that individual studies have limited applicability by themselves in a vacuum to patients with multimorbidity. Guidelines don’t help a they also tend to be single disease focused and developed by single disease -ologists. So is EBM at fault here again? Of course not. EBM skills to the rescue.

The current model of EBM demonstrated below contains 2 important elements: clinical state and circumstances and clinical experience.

Clinical state and circumstances largely refers to the patient’s comorbidities, various other treatments they are receiving, and the clinical setting in which the patient is being seen. Thus, the EBM paradigm is specifically designed to deal with multimorbidity. Clinical expertise is used to discern what impact other comorbidities have on the current clinical question under consideration. and, along with the clinical state/circumstance, helps us decide how to apply a narrowly focused study or guideline in a multimorbid patient. Is this ideal? No. It would be nice if we had studies that included patients with multiple common diseases but we have to treat patients with the best available evidence that we have.

 

 

Standard

Posted by

Posted on

July 27, 2014

Posted under

evidence based practice, General EBM

Comments

Leave a comment

evidence based medicine is not in crisis! part 2

Greenhalgh and colleagues report that the “second aspect of evidence based medicine’s crisis… is the sheer volume of evidence available”. EBM is not the purveyor of what is studied and published. EBM is a set of skills to effectively locate, evaluate, and apply the best available evidence. For much of what we do there is actually a paucity of research data answering clinically relevant questions (despite there being alot of studies- which gets back to her first complaint about distortion of the evidence brand. See part 1 of this series). I teach my students and housestaff to follow the Haynes’ 6S hierarchy when trying to answer clinical questions. As much of the hierarchy is preappraised literature someone else has had to deal with the “sheer volume of evidence”. Many clinical questions can be answered at the top of the pyramid.

I concur with Greenhalgh that guidelines are out of control. I have written on this previously. We don’t need multiple guidelines on the same topic, often with conflicting recommendations. I believe that we would be better off with central control of guideline development under the auspices of an agency like AHRQ or the Institute of Medicine. It would be much easier to produce trustworthy guidelines and guidelines on topics for which we truly need guidance. (Really American Academy of Otolaryngology….do we need a guideline on ear wax removal?) It can be done.  AHCPR previously made great guidelines on important topics. Unfortunately we will probably never go back to the good ole days. Guidelines are big business now, with specialty societies staking out their territory and government and companies bastardizing them into myriad performance measures.

 

 

Standard

Posted by

Posted on

July 25, 2014

Posted under

evidence based practice

Comments

Leave a comment

Evidence Based Medicine Is Not In Crisis! Part 1

Trisha Greenhalgh and colleagues wrote an opinion piece in BMJ recently lamenting (or perhaps exalting) that the EBM movement is in crisis for a variety of reasons. I don’t agree with some of the paper and I will outline in a series of posts why I disagree.

When most people complain about EBM or discuss its shortcomings they usually are not basing their arguments on the current definition of EBM.  They use the original definition of EBM in which EBM was defined as the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. This definition evolved to “the integration of best research evidence with clinical expertise and patient values. Our model acknowledges that patients’ preferences rather than clinicians’ preferences should be considered first whenever it is possible to do so“.

The circles in this diagram are ordered based on importance- with patient preferences and actions being most important and research evidence being the least important when practicing EBM. You can see that clinical expertise is used to tie it all together and decide on what should be done, not what could be done.

Back to the Greenhalgh paper. Her first argument  is that there has been distortion of the evidence brand. I agree. It seems everyone wants to add the “evidence based” moniker to their product. But she argues beyond just a labeling problem. She argues that the drug and medical device industry is determining our knowledge because they fund so many studies. Is this the fault of EBM? Or should funding agencies like the NIH and regulatory agencies like the FDA be to blame? I think the latter. Industry will always be the main funder of studying their product and they should be. They should bear the cost of getting product to market. That is their focus. To suggest they shouldn’t want to make profit is just ridiculous.

The problem arises in what the FDA (and equivalent agencies in other countries) allows pharma to do. Greenhalgh points out the gamesmanship that pharma plays when studying their drug to get the outcomes they desire. I totally agree with what she points out. Ample research proves her points. But it’s not EBM’s fault. The FDA should demand properly conducted trials with hard clinical outcomes be the standard for drug approval. Companies would do this if they had to to get drug to the market. I also blame journal editors who publish these subpar studies. Why do they? To keep advertising dollars? The FDA should also demand that any study done on a drug be registered and be freely available and published somewhere easily accessible (maybe clinical trials.gov). Those with adequate clinical and EBM skills should be able to detect when pharma is manipulating drug dosages, using surrogate endpoints, or overpowering a trial to detect clinically insignificant results. I look at this as a positive for continuing to train medical students and doctors in these skills.

Research has shown that industry funded studies overestimate the benefits of their drugs by maybe 20-30%. A simple way to deal with this is to take any result from an industry funded study and to reduce it by 20-30%. If the findings remain clinically meaningful then use the drug or device.

I agree with Greenhalgh that current methods to assess study biases are outdated. The Users’ Guides served their purpose but need to be redone to detect the subtle gamesmanship going on in studies. Future and current clinicians need to be trained to detect these subtle biases. Alternatively, why can’t journals  have commentaries about every article similar to what BMJ Evidence Based Medicine and ACP Journal Club do. This could then be used to educate journal users on these issues and put the results of studies into perspective.