An interesting article was published in the British Medical Journal in April (http://www.bmj.com/content/346/bmj.f2304.pdf%2Bhtml). Researchers looked at the influence of sample size on treatment effect estimates. The bottom line is that they found that treatment effect estimates were significantly larger from smaller trials as compared to larger trials (up to 48% greater!). The figure below shows this relationship. The left graph compares sample sizes from each study broken into quartiles and on the right arbitrary divisions by raw numbers.
So what does this mean for the average reader of medical journals? Pay attention to sample size. Early studies on new technology (whether meds or procedures) are often carried out on a fairly small group of people. Realize that what you see is likely overestimated (compared to a large study). If benefits are marginal (barely clinically significant) realize they likely will go away with a larger trial. If benefits are too good to be true….they likely are too good to be true and you should temper your enthusiasm. I always like to see more than one trial on a topic before I jump in a prescribe new meds or recommend new procedures.
I began developing an online course in EBM and use videos for instruction. I thought it would be useful to tell others what I use. Videos are a useful way to teach others and are more learner-centered in that learners can use them when they want and can skip around in them for the information they need.
First off your videos need to be relatively short: 3-5 minutes is optimal. Longer than 10 minutes and learners will abandon the video. You have to be thoughtful on how to break up a longer topic into shorter chunks. They need to be engaging either through your teaching style or material presentation. They need to be consistent- use the same backgrounds, intro and exits.
Here’s what you need to make teaching videos: (Note: I am a PC guy. Mac has good built-in software that I can’t really comment on as I dont use Macs) 1) Video Hosting Service: YouTube and Vimeo are 2 of the more popular hosting sites. Some of the tools below also have their own site to post videos on. I personlly like YouTube. Here is my YouTube channel: (http://www.youtube.com/user/UABEBMcourse/videos?view=0)
2) Video Editing Software: I actually record myself giving intros and endings for each of my teaching videos. That way I have a “physical” presence. Because of this I need a video editor to splice these together with the main body of my video. There’s lots of video editing software out there. I use Corel Vidoe Studio Pro because it also contains a screen capture tool and seemed easy to use and wasnt too expensive ($60). (http://www.corel.com/corel/product/index.jsp?pid=prod4900075&cid=catalog20038&segid=8100022&storeKey=us&languageCode=en)
3) Screen Capture Software: I usually display a PowerPoint slide or a paper or a website when I teach. I use sceen capture software to record my computer screen and my voice. I like Screencast-o-matic (www.screencast-o-matic.com) because its easy and cheap (free). I bought the Pro version ($15/yr) so I have more tools available but the free version is good.
4) Video camera or webcam if you want to video something other than your computer screen.
5) Bamboo tablet: I use this to write on my screen when I explain/demonstrate how to make calculations. (http://www.wacom.com/en/products/pen-tablets/bamboo). This YouTube video shows me using the tablet and the digital drawing program I describe below:
7) Microphone. Often they are built in to your computer or webcam. I bought a high def one off ebay for <$10 because I wanted improved sound.
8) Audacity (http://audacity.sourceforge.net/) is software for recording and editing sound. If you want to just do voice recordings this is a good program.
So that’s it. For a minimal outlay of money you can develop pretty good quality videos. There is a learning curve on editing. You will also have to decide your style. Look around at other videos and do what works for you. This guy (http://www.youtube.com/user/bozemanbiology?feature=watch) has incredible videos but they take a little more time and skill to make.
Many will say this is a stupid question. Of course they can’t be open access because the journal needs to make money to exist. But think about this: what is the point of a published medical journal article?
Isn’t knowledge dissemination the main point of a journal article? So why do we not have access to all journal articles?
How would journals continue to exist without subscriptions and advertising dollars? The model for most open access journals is that the authors pay for their article to be published. That is a viable option. Another option is that journals could develop special content that would still attract subscribers and not be open access. For example, JAMA has great content about the clinical examination and how to read research articles. Journals could develop apps for practice based application of material. Etc. Etc.
What about advertisers? Why would they continue to take out ads. Think about open access. Even more people would see the ads. As part of the “right” to download an article a little advertising banner is printed on the article somewhere. Thus, advertiser exposure is increased even more.
I don’t pretend to have all the answers but it seems that medical knowledge should be in the hands of those that need it, when they need it. The most important information gets published in the most restrictive journals. I think its time for the development of a creative way to fund open access to all journals.
I am a member of an international listserv about evidence-based healthcare. One poster asked “Is EBM patient-centered and is patient-centered care evidence based?” It is almost as if he views the 2 as exclusionary. In my experience many people don’t understand the EBM paradigm. This figure shows what EBM is and that it,by definition, is patient-centered.
The most important component of the EBM paradigm (the circles are in the order of importance) is patient preferences and actions. An evidence-based decision should consider patient values. Period. Thus, EBM is patient-centered.
Question 2: Is patient-centered care evidence based? It can be but might not be. Patients often don’t want the evidence-based care I offer them like immunizations or colon cancer screening. So they aren’t receiving evidence-based care but they are receiving patient-centered care.
I am reviewing a manuscript about the trustworthiness of guidelines for a prominent medical journal. I have written editorials on this topic in the past (http://jama.jamanetwork.com/article.aspx?articleid=183430 and http://archinte.jamanetwork.com/article.aspx?articleid=1384244). The authors of the paper I am reviewing reviewed the recommendations made by 3 separate medical societies on the use of a certain medication for patients with atrial fibrillation. The data on this drug can be summarized as follows: little benefit, much more harm. But as you would expect these specialists recommended its use in the same sentence as other safer and more proven therapies. They basically ignored the side effects and only focused on the minimal benefits.
Why do many guideline developers keep doing this? They just can’t seem to develop guidelines properly. Unfortunately their biased products have weight with insurers, the public, and the legal system. The reasons are complex but solvable. A main reason (in my opinion) is that they are stuck in their ways. Each society has its guideline machine and they churn them out the same way year after year. Why would they change? Who is holding them accountable? Certainly not journal editors. (As a side note: the journals that publish these guidelines are often owned by the same subspecialty societies that developed the guidelines. Hmmmm. No conflicts there.)
The biggest problem though is conflicts of interest. There is intellectual COI. Monetary COI. Converting data to recommendations requires judgment and judgment involves values. Single specialty medical society guideline development panels involve the same types of doctors that have shared values. But I always wonder how much did the authors of these guidelines get from the drug companies? Are they so married to this drug that they don’t believe the data? Is it ignorance? Are they so intellectually dishonest that they only see benefits and can’t understand harm? I don’t think we will ever truly understand this process without having a proverbial fly on the wall present during guideline deliberations.
Until someone demands a better job of guideline development I still consider them opinion pieces or at best consensus statements. We need to quit placing so much weight on them in quality assessment especially when some guidelines, like these, recommend harmful treatment.
A colleague of mine (DB) has a great blog (medrants.com). He posted (http://www.medrants.com/archives/7199) an excellent observation that ABIM pass rates are declining since the work hour restrictions have gone into place. I commented back that why would we expect any different because systems give us exactly what they are supposed to give us. In this case less knowledgeable residents.
The proof is in the scores. Why? My theory is that they waste their off time with being off and not studying. When I was a resident in the mid 90s we had no restrictions, no admission caps and only 1 guaranteed day off a month. We stayed at the hospital until late afternoon at least if not early evening routinely. We studied alot during the day in our down time. The satellite library at the hospital was full of us studying (and yes BSing). I saw alot more patients and did alot more to them than this current generation. Interestingly I dont recall mass killings and mayhem at my training hospitals. But for some reason we have accepted a system (work hour restrictions and admission caps) without testing it ahead of time. No drug gets to market this way but for some reason its ok to let doctors get to market this way.
Being a doctor is complex. Clinical reasoning relies on matching the patient’s story to an illness script. You build illness scripts from studying books (print or electronic…doesnt matter) and by patient interactions. You have to see lots of patients to enhance and mature your illness scripts. Therein lies the problem…..less studying and less patients. Wow…how did we ever get these lower passing rates. Its hard to understand (sarcasm detected). Oh well it doesnt matter. I am sure the wonderfully designed quality measures will make it all better.
Recently I got interesting in more formalized teaching of clinical reasoning. I am currently enrolled in a Coursera (coursera.org) course on clinical reasoning. My reason for taking this MOOC is that I wanted to see how others teach this material so that I could improve my teaching. A great revelation I had while reading Judith Bowen’s now classic NEJM article () and which was reenforced in the Coursera course was vertical reading. The way I summarize this is that we recognize diseases best, not by their similarities, but by their differences. With vertical reading you compare 2 or more diseases by filing out a table with epidemiology, pathophysiology, signs, symptoms, testing, etc. Instead of reading about 1 disease at a time and filling in the table horizontally you fill in the table vertically and read about each component of the table for both diseases at the same time.
The reason I bring this topic up is that I am doing journal club next week and I have asked the residents to read 2 articles on seemingly the same topic. There are nuances that make the articles different but if you read them individually and separately you likely wouldn’t pick up on these nuances. Thus, the vertical reading. I gave the residents a table to fill out comparing the 2 articles. They will read them both at the same time comparing the articles on each of the elements of the table;moving vertically down the table. They should be able to detect the nuances of each article. We will see.
It struck me today as was listening to the radio and reviewing an email summary of articles that I get every day that 2 “quality” measures that I am held accountable for don’t really help much. I am talking about this year’s influenza vaccine and the pneumococcal vaccine.
Lets start with the influenza vaccine. Where I practice I am supposed to make at least 79% of my patients over age 65 yrs of age take the vaccine. This doesn’t sound too bad right? Why shouldn’t it be 100%? Well the problem this year is that the influenza vaccine mostly sucks in this age group….per the CDC its only 9% effective in persons 65 yrs of age and older . (http://www.cdc.gov/MMWR/preview/mmwrhtml/mm6207a2.htm?s_cid=mm6207a2_w)
What about the pneumococcal vaccine? It should really help people right? I am supposed to make 95% of my patients take this vaccine. Well it kind a sucks also per Moberley S, Holden J, Tatham DP, et al. Vaccines for preventing pneumococcal infection in adults. Cochrane Database Syst Rev. 2012 Jun 22;1:CD000422. DOI: 10.1002/14651858.CD000422.pub3. This well done Cochrane review found that invasive pneumococcal disease was prevented by the vaccine but not pneumonia or mortality. So all it really prevents is bacteremia if you get pneumococcal pneumonia but it actually doesn’t prevent pneumonia. Somewhat of a misnamed vaccine if you ask me.
There is little benefit of these 2 vaccines but yet I am supposed to recommend them to my patients. I don’t have great vaccination rates in my patients. Maybe I won’t feel so bad about that any longer. The policy wonks who make up these rules need to look at the data and measure what’s important. Unfortunately they don’t. They are mired in their measurement mentality without the benefit of an intellect.
