Category Archives: General EBM

Standard

Posted by

Posted on

July 28, 2014

Posted under

evidence based practice, General EBM

Comments

3 Comments

Evidence Based Medicine Is Not In Crisis! Part 3

In this installation I want to jump ahead in Greenhalgh’s paper to address her last cause of the EBM crisis: “Poor fit for multimorbidity“. Not to worry, I will come back in a future post to cover the remaining “problems” of EBM.

I concur with Greenhalgh that individual studies have limited applicability by themselves in a vacuum to patients with multimorbidity. Guidelines don’t help a they also tend to be single disease focused and developed by single disease -ologists. So is EBM at fault here again? Of course not. EBM skills to the rescue.

The current model of EBM demonstrated below contains 2 important elements: clinical state and circumstances and clinical experience.

Clinical state and circumstances largely refers to the patient’s comorbidities, various other treatments they are receiving, and the clinical setting in which the patient is being seen. Thus, the EBM paradigm is specifically designed to deal with multimorbidity. Clinical expertise is used to discern what impact other comorbidities have on the current clinical question under consideration. and, along with the clinical state/circumstance, helps us decide how to apply a narrowly focused study or guideline in a multimorbid patient. Is this ideal? No. It would be nice if we had studies that included patients with multiple common diseases but we have to treat patients with the best available evidence that we have.

 

 

Standard

Posted by

Posted on

July 27, 2014

Posted under

evidence based practice, General EBM

Comments

Leave a comment

evidence based medicine is not in crisis! part 2

Greenhalgh and colleagues report that the “second aspect of evidence based medicine’s crisis… is the sheer volume of evidence available”. EBM is not the purveyor of what is studied and published. EBM is a set of skills to effectively locate, evaluate, and apply the best available evidence. For much of what we do there is actually a paucity of research data answering clinically relevant questions (despite there being alot of studies- which gets back to her first complaint about distortion of the evidence brand. See part 1 of this series). I teach my students and housestaff to follow the Haynes’ 6S hierarchy when trying to answer clinical questions. As much of the hierarchy is preappraised literature someone else has had to deal with the “sheer volume of evidence”. Many clinical questions can be answered at the top of the pyramid.

I concur with Greenhalgh that guidelines are out of control. I have written on this previously. We don’t need multiple guidelines on the same topic, often with conflicting recommendations. I believe that we would be better off with central control of guideline development under the auspices of an agency like AHRQ or the Institute of Medicine. It would be much easier to produce trustworthy guidelines and guidelines on topics for which we truly need guidance. (Really American Academy of Otolaryngology….do we need a guideline on ear wax removal?) It can be done.  AHCPR previously made great guidelines on important topics. Unfortunately we will probably never go back to the good ole days. Guidelines are big business now, with specialty societies staking out their territory and government and companies bastardizing them into myriad performance measures.

 

 

Standard

Posted by

Posted on

June 15, 2014

Posted under

Bias, General EBM, Study design

Comments

Leave a comment

Useful diagram to teach basic EBM concepts

Dr. La Rochelle published an article in BMJ EBM this month with a very useful figure in it (see below). It is useful because it can help our learners (and ourselves) remember the relationship between the type of evidence and its believability/trustworthiness.

Untitled

Lets work through this figure. The upright triangle should be familiar to EBM aficionados as it is the typical hierarchy triangle of study designs, with lower quality evidence at the bottom and highest quality at the top (assuming, of course, that the studies were conducted properly). The “Risk of Bias” arrow next to this upright triangle reflects the quality statement I just made.  Case reports and case series, because they have no comparator group and aren’t systematically selected are at very high risk of bias. A large RCT or systematic review of RCTs is at the lowest risk of bias.

The inverted triangle on the left reflects possible study effects, with the width of the corresponding area of the triangle (as well as the “Frequency of Potential Clinically relevant observable effect arrow) representing the prevalence of that effect. Thus, very dramatic, treatment altering effects are rare (bottom of triangle, very narrow). Conversely, small effects are fairly common (top of triangle, widest part).

One way to use this diagram in teaching is to consider the study design you would choose (or look for) based on the anticipated magnitude of effect. Thus, if you are trying to detect a small effect you will need a large study that is methodologically sound. Remember bias is a systematic error in a study that makes the findings of the study depart from the truth. Small effects seen in studies lower down the upright pyramid are potentially biased (ie not true). If you anticipate very large effects then observational studies or small RCTs might be just fine.

An alternative way to use this diagram with learners is to temper the findings of a study. If a small effect is seen in a small, lower quality study they should be taught to question that finding as likely departing from the truth. Don’t change clinical practice based on it, but await another study. A very large effect, even in a lower quality study, is likely true but maybe not as dramatic as it seems (ie reduce the effect by 20-30%).

I applaud Dr. La Rochelle for developing a figure which explains these relationships so well.

Standard

Posted by

Posted on

May 23, 2014

Posted under

General EBM, journal club

Comments

1 Comment

EBM Rater is finally available

I have always suspected that one reason that physicians don’t critically appraise articles is that the criteria for critical appraisal are not readily available in a convenient, easy to use package. No more. I, with the help of some undergraduate computer science students, have created a critical appraisal app for Android devices. Its in the Google playstore and will be listed in the Amazon app store. Hopefully will develop an iOS version if this version is successful.

 

screen shot

I tried to take critical appraisal to the next step by “scoring” each study and giving an estimate of the bias in the study. I then make a recommendation of whether or not the user should trust the study or reject it and look for another study. I think one of the shortcomings of the Users’ Guides series is that no direction is given to the user about what to do with the article after you critically appraise it. EBM Rater will give a suggestion about the trustworthiness of the study.

EBM Rater contains criteria to critically appraise all the major study designs including noninferiority studies. It even contains criteria to evaluate surrogate endpoints, composite endpoints, and subgroup effects.
screen shot

Finally, it contains standard EBM calculators like NNT, NNH, and posttest probability. I added 2 unique calculators that I have not seen in any other app: patients specific NNT and NNH. Many of our patients are sicker or healthier that the patients included in a study.  NNTs and NNHs are typically calculated with data from a study so the NNT and NNH is for the study patients. With my calculator you can figure out your individual patient’s NNT or NNH.

screen shot

I hope you will give it a try and give me some feedback.