Advanced EBM Elective for Medical Students

I teach an advanced EBM co-enrolled elective in the Spring semester to 3rd and 4th-year medical students.Below is a screen shot of the course home page.

advanced-ebm-screenshot

This year I decided to revamp it completely and make it more philosophical. It was a challenge to decide what topics to include. You can see the topics I plan to cover along the right side of the image. I wanted them to be thought provoking and also useful at some level and not something taught in the usual EBM course.  I also made some other big changes after being inspired by 2 classes (Introduction to Openness and Social Network Learning) I recently took as part of my Master of Educational Technology degree (Thanks  Fred Baker and Jackie Gerstein). Here are the changes:

  • It’s open (all materials are free to use and anyone can take the course at any time). I hope to get some non-UAB students to take the course at the same time as my UAB students- a mini-MOOC so to speak.
  • Students will use social network tools extensively to enhance learning.
  • Students get a say in what they learn. I have designed the learning modules but I encourage students to develop their own learning module to replace one of the ones I developed.
  • Students will learn the value of social media for finding, creating, and sharing information.
  • Students will learn about personal learning networks and how to cultivate them.
  • Students will be exposed to learning activities they probably haven’t been exposed to in the past or used very much (e.g. jigsaw activity, curation,  concept mapping, blogging, tweeting) in medical education.

I hope these changes will enhance the ability of students to make meaning of this material. If nothing else I enjoyed creating a new class.

EBM is just not a priority in medical education

When I reflect on what I do each day as a physician it occurs to me that I use EBM skills very commonly. Here is a sampling:

  • I think about and assess pretest probability a lot
  • I think about choosing appropriate tests a lot
  • I apply information from studies a lot. I weigh risks and benefits of therapies. I think about patient context. I try to incorporate patient values and desires as much as possible.
  • I search for information following the Haynes’ 6S approach
  • I critically appraise primary studies and systematic reviews each week (not daily)
  • I make calculations because studies don’t always put information in the format I want
  • I have discussions with patients about the above issues

I am sure I am missing a lot of what I do that falls under “EBM”. I am revamping an introductory course in EBM I teach to 2nd medical students for the upcoming semester. It has been relegated to “just teach them enough to get a good score on Step 1”. Thankfully, I have a fuller online version that they will take during their scholarly time in the 3rd year so all is not lost. To make me feel better, I view the crash course I am teaching them this upcoming semester as scaffolding so that they can better understand my full online course. You can look at and use the materials I will use in the crash course in the tab above labeled “Online Teaching Resources” (I just realized still have to add a few items that the students will use).

We spend so much time in the 1st 2 years of medical school teaching about things that I honestly never ever use but yet what I use daily gets short shrift. Why is that? Are EBM skills not important? Is it assumed they are easy to develop later in one’s career on one’s own (they aren’t)? Is it just kicking the can down the road assuming in residency these skills will be learned? Or during the clerkships?

I for one wish none of this material was on Step 1. I think it’s too early. Furthermore, I am so sick of my course evaluations including statements like “Taught too much stuff that wasn’t on step 1”. I think you need some clinical knowledge to really learn EBM, but more importantly, to understand its importance. EBM type questions should get greater prominence on Step 2 and even more prominence on step 3 exams. One or 2 questions only reinforces the perceived lack of importance of EBM. EBM should have just as many questions as any of the specialties and each test should have more questions to reinforce that these skills are important and will be used. Maybe Santa will grant me that wish one of these years. (I am keeping my fingers crossed I get onto the NBME committee that writes the EBM questions. Maybe I can convince them of my plan)

 

 

I wonder how much EBM is really practiced out there

WARNING: a lot of cynicism in this post.

I have been revamping my EBM course that I teach at the medical school. As I’ve been doing this I realize we (the collective EBM teachers of the world) teach knowledge and skills that I don’t think are used very often once our doctors are out of residency.

Who really develops a PICO question in the clinical setting (outside of an academic center)?  Who is really doing database searches? (I think everyone just goes to Google, UpToDate or Dynamed and doesn’t care if studies are potentially missed.) How many critically appraise the primary literature? (Don’t most probably just read the conclusions from the abstract? or assume the study is good?) How many really understand how to “manipulate” findings of a study to adapt them to the patient they are seeing?

I know this seems like a negative post but practicing EBM is hard. It is a complex task that takes time and feedback to master. Once you leave training there is little feedback you will ever get on EBM skills. So they wane and all that can be done is to keep practicing like they have been by relying on experience, collective knowledge of consultants, and using Dr. Google. But how bad of a service have they provided their patients by doing this? Probably not all that bad.

As an educator I feel these skills are important and I think I have designed my course to provide the best chance for students to remember the material. But I don’t know how to convince practicing docs that they need to keep brushing up on EBM skills. I also don’t know what I would tell them if they asked “Well how do you want me to brush up on my EBM skills?” EBM skills should probably be a reasonably important part of the MOC process. Aren’t these skills key to actually keeping up?

Now its your turn. Tell me where I’m wrong and what should practicing docs do?

3 Pronged Approach to Reading a Clinical Study

There has been an interesting discussion on how to critically appraise a study on the Evidence-Based Health listserv over the last day. It is interesting to see different opinions on the role of critical appraisal.

One important thing to remember as pointed out by Ben Djulbegovic is that critical appraisal relies on the quality of reporting as these 2 studies showed: http://www.ncbi.nlm.nih.gov/pubmed/22424985  and http://www.ncbi.nlm.nih.gov/pmc/articles/PMC313900/ . The implications are important but difficult for the busy clinician to deal with.

There are 3 questions you should ask yourself as you read a clinical study:

  1. Are the findings TRUE?
  2. Are the findings FREE OF THE INFLUENCE OF BIAS?
  3. Are the findings IMPORTANT?

The most difficult question for a clinician to answer initially is if the findings are TRUE. This question gets at issues of fraud in a study. Thankfully major fraud(ie totally fabricated data) is a rare occurrence.  Totally fraudulent data usually gets exposed over time. Worry about fraudulent data when the findings seem too good to be true (ie not consistent with clinical experience). Usually other researchers in the area will try to replicate the findings and can’t. There are other elements of truth that are more subtle and occur more frequently. For example, did the authors go on a data dredging expedition to find something positive to report? This would most commonly occur with post hoc subgroup analyses. These should always be considered hypothesis generating and not definitive. Here’s a great example of a false subgroup finding:

The Second International Study of Infarct Survival (ISIS-2) investigators reported an apparent subgroup effect: patients presenting with myocardial infarction born under the zodiac signs of Gemini or Libra did not experience the same reduction in vascular mortality attributable to aspirin that patients with other zodiac signs had.

Classical critical appraisal, using tools like the Users’ Guides, is done to DETECT BIASES in the design and conduct of studies. If any are detected then you have to decide the degree of influence that the bias(es) has had on the study results. This is difficult, if not impossible, to determine with certainty but there are studies that estimate the influence of various biases (for example, lack of concealed allocation in a RCT) on study outcomes. Remember, most biases lead to overestimate of effects. There are 2 options if you detect biases in a study: 1) reduce the “benefit” seen in the study by the amounts demonstrated in the following table and then decide if the findings are still important enough to apply in patient care, or 2) discard the study and look for one that is not biased.

This table is synthesized from findings reported in the Cochrane Handbook (http://handbook.cochrane.org/chapter_8/8_assessing_risk_of_bias_in_included_studies.htm)

BIAS                                                                       EXAGGERATION  OF EFFECT OF BENEFIT

Lack of randomization                                                             25% (-2 to 45%)

Lack of allocation concealment                                              18% (5 to 29%)

Lack of blinding                                                                          9% (NR)

Finally, if you believe the findings are true and are free of significant bias,  you have to decide if they are CLINICALLY IMPORTANT. This requires clinical judgment and understanding the patient’s baseline risk of the bad outcome the intervention is trying to impact. Some people like to calculate NNTs to make this decision. Don’t just look at the relative risk reduction and be impressed because you can be misled by this measure as I discuss in this video: https://youtu.be/7K30MGvOs5s

Workshop on Developing Open Educational Resources

On Friday September 23, 2016 I am presenting a workshop on developing open educational resources (OERs) at the UAB Research and Innovations in Medical Education conference.

This Hyperdoc is a self-guided version of the workshop.

These are the Google slides I will use at the presentation.

I became very interested in openness during recent coursework for my Master in Educational Technology degree. I blog about my experiences in that course here.

If you want a good overview of openness download The Battle for Open ebook by Martin Weller.

Now it’s your turn: Tell me what you think of the materials or open resources/learning/publishing in general.