Author Archives: Terry Shaneyfelt

Standard

Posted by

Posted on

August 13, 2013

Posted under

journal club, statistics

Comments

Leave a comment

Journal Club- Basic Stats: Answers

Here are my answers to the journal club questions. I have also added links to some of my youtube videos to answer questions

1) The authors designed the study to have a “power of more than 80%“. What does this mean?
Power is the probability of the study finding a difference given that one truly exists. So this study was designed with at least an 80% chance of finding a difference between treatment and control groups (given that one truly exists). This video explains power in a little more depth.
2) What was the planned type 1 error rate in this study? Type 1 error is also called the alpha error. They planned on a 5% type 1 error rate. This video explains type 1 error in a little more detail
3) What is a type 2 error and how is it related to power? Type 2 error is also called beta error. It is related to power in that power is 1 (or 100%) minus the beta error. So if power is 80% the type 2 error rate is 20%. This video  explains type 2 error in more detail.
4) What are the determinants of sample size in this study? How does varying the estimates of these components affect sample size? Sample size is determined by a variety of factors: power, type 1 and 2 error rates, estimated difference between study groups and variability in the data (though this last one has less of an effect). See this video explaining these factors and their effect on sample size.
5) The authors use a variety of statistical tests (chi-square, Fisher’s exact, t-tests, etc) to analyze the data. In general, what do statistical tests do?
Statistical tests look at the data and calculate a test statistic (e.g. t statistic for a t test). The test statistic is then used to determine the p-value assosicated with the data.

Review Table 2 and answer the following questions:1) The primary outcome occurred in 1.92/100 person-yrs in the control group compared to 1.83/100 person-yrs in the intervention group. The p-value associated with this comparison is 0.51. What does this p-value mean? Can p-values be used to detect bias in the study? The simple interpretation is that the difference is not statistically significant because the p-value is > 0.05. Another interpretation would be that the difference seen between the groups or one more extreme is due 51% likely due to chance. P-values cannont detect bias (systematic errors) in a study. Critical appraisal detects bias.
2) The hazard ratio comparing the intervention group to the control group for the primary outcome is 0.95 with a 95% confidence interval of 0.83-1.09. What does this confidence interval tell you about the effect? Can confidence intervals be used to detect bias in the study? It tells you a couple of things: 1. that the difference is not statistically significant as the CI included the point of no difference…1.0 and 2. that the benefit could be up to 17% reduction in cardiovascular events or 9% increase. This video explains how to interpret hazard ratios and this video confidence intervals.

Finally the extra credit: These 4 things can explain study findings: truth, chance, bias, confounding

I hope this was somewhat helpful. I will have another journal club next month on another EBM topic.

Standard

Posted by

Posted on

July 31, 2013

Posted under

journal club, statistics

Comments

Leave a comment

Journal Club- Basic Stats: Cardiovascular Effects of Intensive Lifestyle Intervention in Type 2 Diabetes

I decided to start a new feature that hopefully you and I will find useful. It will only be useful if you work thru the questions and have a dialogue thru the comments section.

My plan is to post a different article about once a month and have questions for you to answer. 1 week later (or so) I will then either post a video review of my answers or just write about the answers. I plan to focus mostly on the basics but at times I will cover advanced topics also as “extra credit”. I am mostly going to parallel the journal club curriculum we are using this year at UAB. I welcome comments to make this better or articles you want to read.

journal club

58 yo M with DM-2, hyperlipidemia and HTN presents to you for a follow-up visit. He takes metformin 1000mg BID, lisinopril 20mg daily, and pravastatin 40mg nightly. His most recent HgA1C was 6.9% and LDL was 88 mg/dl. In the office his blood pressure is 128/67 mm Hg and BMI is 32. You counsel him to lose weight and he responds “My blood pressure, cholesteol and A1C are good. How is losing weight going to help my heart?” What do you tell him?

Article: The Look AHEAD Research Group. Cardiovascular Effects of Intensive Lifestyle Intervention in Type 2 Diabetes. NEJM 2013;369:145-54.

After reading the statistical analysis section (pgs 147-148) of the article answer the following questions:
1) The authors designed the study to have a “power of more than 80%“. What does this mean?
2) What was the planned type 1 error rate in this study?
3) What is a type 2 error and how is it related to power?
4) What are the determinants of sample size in this study? How does varying the estimates of these components affect sample size?
5) The authors use a variety of statistical tests (chi-square, Fisher’s exact, t-tests, etc) to analyze the data. In general, what do statistical tests do?

Review Table 2 and answer the following questions:1) The primary outcome occurred in 1.92/100 person-yrs in the control group compared to 1.83/100 person-yrs in the intervention group. The p-value associated with this comparison is 0.51. What does this p-value mean? Can p-values be used to detect bias in the study?
2) The hazard ratio comparing the intervention group to the control group for the primary outcome is 0.95 with a 95% confidence interval of 0.83-1.09. What does this confidence interval tell you about the effect? Can confidence intervals be used to detect bias in the study?

Extra Credit:
List the 4 things that can explain study findings

Standard

Posted by

Posted on

July 12, 2013

Posted under

General information

Comments

Leave a comment

Finally Someone Has Some Sense: Patient-Centered Quality Assessment

This week in JAMA Eve Kerr and Rodney Heyward wrote an important opinion piece about patient centered performance measurement. So here’s the problem with our current performance measurement paradigm: one size fits all. Guidelines have been bastardized into performance measures. Administrators/insurers/etc then hold us accountable to the measures. So what’s the problem?

Well guidelines were meant to be just that guidelines, not rules. When they are made into performance measures they become rules. Rigid rules that are applied to very variable patients. Deviation from the rules leads to the clinician being “dinged”. What this often leads to is inappropriate care. Patients have medication regimens inappropriately escalated to reach an arbitrary number. This often leads to harm either directly as an effect of that medication or through medication interactions. Patients get tests they don’t need (for example, colon cancer screening in patients with multiple comorbidities and limited life expectancy).

Quality graph

I have always felt that quality is very personal. What I view as quality will not be the same as someone else. This applies to all aspects of life, not just medicine. I have always felt that we should negotiate with our patients for what constitutes quality care FROM THEIR STANDPOINT. Aren’t they the ones who matter most? Why is it some administrator decides what constitutes quality care?

Let’s look at an example. Suppose I have a patient with diabetes who is on maximal oral therapy but her A1C is 9. It would be recommended to give insulin. What if the patient doesn’t want insulin no matter what. She understands the risk of longterm uncontrolled insulin. Is it poor quality care that I honor her request to only stay on oral therapy? Again from whose perspective should we evaluate quality?

I think we are slowly moving to including the patient in quality assessment. It needs to go further to patients being actively involved in developing their own quality report card. Then we can be measured for how well we meet quality.

Standard

Posted by

Posted on

June 17, 2013

Posted under

Uncategorized

Comments

1 Comment

Focus On The Body Of Evidence…Not Just Individual Studies

JAMA published an important article last week. This is an important article because it reminds us to review the totality of the evidence and not just an individual study. There are several reasons to focus on the whole body of evidence and not just individual studies:
– So as not to be misled by an individual publication. Individual studies are subject to publication bias- the disproportionate publication of positive studies to the exclusion of negative studies. Individual studies are also just one of a distribution of findings.
– Evaluate the consistency of the effect. Is the magnitude of the effect similar across studies? If so this is reassuring. If not you will have to figure out why and what it means.
– Individual studies can be flawed/biased and a body of evidence is less likely to be flawed (though it could be).
– Multiple small underpowered studies can become more useful when combined (ie metaanlysis). If you only read the one underpowered study you might conclude that the intervention is useless.

A colleague of mine has recently changed his prescribing habits for diuretics in the treatment of hypertension. He based this decision on the recent publication of a single study comparing hydrochlorothiazide to chlorthalidone. When I examined the totality of the evidence (2 meta-analyses I come to a different conclusion than he did (I will be posting a blog on this in the near future).

So what does this mean as you try to answer a clinical question? What about when you are just reading through your weekly journal?
1) Use synthesized evidence first to answer questions. These are higher up in the Haynes’ hierarchy (the figure below) and included evidence based textbooks like Dynamed. Second line would be synopses of systematic reviews (called syntheses in the Haynes’ hierarchy). Next would be a systematic review (synopsis).
Haynes' 6S Hierarchy

2) After reading an individual study look for systematic reviews on the same intervention. Authors will sometimes refer to them in the intro and discussion sections of their paper. Look to see how the current paper compares to the rest of the evidence so you can put the new findings in perspective.

The limitation of this approach is time (although reading an individual study actually takes more time than reading a synthesis) and access to resources. There is no reason for a practicing clinician not to have access to an evidence based resource like Dynamed (or UpToDate- though I hesitantly put this here for my own reasons). Costs of these resources is reasonable and is money better spent than journal subscriptions.

Standard

Posted by

Posted on

May 6, 2013

Posted under

Bias, Study results

Comments

Leave a comment

Small Studies Can Lead To Big Results

An interesting article was published in the British Medical Journal in April. Researchers looked at the influence of sample size on treatment effect estimates. The bottom line is that they found that treatment effect estimates were significantly larger from smaller trials as compared to larger trials (up to 48% greater!). The figure below shows this relationship. The left graph compares sample sizes from each study broken into quartiles and on the right arbitrary divisions by raw numbers.

Comparison of treatment effect estimates between trail sample sizes

So what does this mean for the average reader of medical journals? Pay attention to sample size. Early studies on new technology (whether meds or procedures) are often carried out on a fairly small group of people. Realize that what you see is likely overestimated (compared to a large study). If benefits are marginal (barely clinically significant) realize they likely will go away with a larger trial. If benefits are too good to be true….they likely are too good to be true and you should temper your enthusiasm. I always like to see more than one trial on a topic before I jump in and prescribe new meds or recommend new procedures.

Standard

Posted by

Posted on

April 19, 2013

Posted under

Education design

Comments

Leave a comment

Tools I Use to Make My Teaching Videos

I began developing an online course in EBM and use videos for instruction. I thought it would be useful to tell others what I use. Videos are a useful way to teach others and are more learner-centered in that learners can use them when they want and can skip around in them for the information they need.

First off your videos need to be relatively short: 3-5 minutes is optimal. Longer than 10 minutes and learners will abandon the video. You have to be thoughtful on how to break up a longer topic into shorter chunks. They need to be engaging either through your teaching style or material presentation. They need to be consistent– use the same backgrounds, intro and exits.

Here’s what you need to make teaching videos: (Note: I am a PC guy. Mac has good built-in software that I can’t really comment on as I dont use Macs)
1) Video Hosting Service: YouTube and Vimeo are 2 of the more popular hosting sites. Some of the tools below also have their own site to post videos on. I personlly like YouTube. Here is my YouTube channel.

2) Video Editing Software: I actually record myself giving intros and endings for each of my teaching videos. That way I have a “physical” presence. Because of this I need a video editor to splice these together with the main body of my video. There’s lots of video editing software out there. I use Corel Vidoe Studio Pro because it also contains a screen capture tool and seemed easy to use and wasnt too expensive ($60).

3) Screen Capture Software: I usually display a PowerPoint slide or a paper or a website when I teach. I use sceen capture software to record my computer screen and my voice. I like Screencast-o-matic because its easy and cheap (free). I bought the Pro version ($15/yr) so I have more tools available but the free version is good.

4) Video camera or webcam if you want to video something other than your computer screen.

5) Bamboo tablet: I use this to write on my screen when I explain/demonstrate how to make calculations. (http://www.wacom.com/en/products/pen-tablets/bamboo). This YouTube video shows me using the tablet and the digital drawing program I describe below.

6) Digital Drawing Program: this is the background I write (or draw) on with the Bamboo tablet. I like Autodesk Sketchbood Express because it was more intuitive to use and free. Khan Academy uses SmoothDraw but I didn’t find it as easy to use.

7) Microphone. Often they are built in to your computer or webcam. I bought a high def one off ebay for <$10 because I wanted improved sound.

8) Audacity (http://audacity.sourceforge.net/) is software for recording and editing sound. If you want to just do voice recordings this is a good program.

So that’s it. For a minimal outlay of money you can develop pretty good quality videos. There is a learning curve on editing. You will also have to decide your style. Look around at other videos and do what works for you. This guy has incredible videos but they take a little more time and skill to make.

Standard

Posted by

Posted on

April 15, 2013

Posted under

Evidence Resources

Comments

Leave a comment

Why Aren’t All Journals Open Access?

Many will say this is a stupid question. Of course they can’t be open access because the journal needs to make money to exist. But think about this: what is the point of a published medical journal article?

Isn’t knowledge dissemination the main point of a journal article? So why do we not have access to all journal articles?

knowledge

How would journals continue to exist without subscriptions and advertising dollars? The model for most open access journals is that the authors pay for their article to be published. That is a viable option. Another option is that journals could develop special content that would still attract subscribers and not be open access. For example, JAMA has great content about the clinical examination and how to read research articles. Journals could develop apps for practice based application of material. Etc. Etc.

What about advertisers? Why would they continue to take out ads. Think about open access. Even more people would see the ads. As part of the “right” to download an article a little advertising banner is printed on the article somewhere. Thus, advertiser exposure is increased even more.

I don’t pretend to have all the answers but it seems that medical knowledge should be in the hands of those that need it, when they need it. The most important information gets published in the most restrictive journals. I think its time for the development of a creative way to fund open access to all journals.

What do you think? What are your solutions?

Standard

Posted by

Posted on

April 4, 2013

Posted under

evidence based practice

Comments

Leave a comment

“Can EBM Be Patient-Centered?”

I am a member of an international listserv about evidence-based healthcare. One poster asked “Is EBM patient-centered and is patient-centered care evidence based?” It is almost as if he views the 2 as exclusionary. In my experience many people don’t understand the EBM paradigm. This figure shows what EBM is and that it,by definition, is patient-centered.

EBM paradigm

The most important component of the EBM paradigm (the circles are in the order of importance) is patient preferences and actions. An evidence-based decision should consider patient values. Period. Thus, EBM is patient-centered.

Question 2: Is patient-centered care evidence based? It can be but might not be. Patients often don’t want the evidence-based care I offer them like immunizations or colon cancer screening. So they aren’t receiving evidence-based care but they are receiving patient-centered care.

Standard

Posted by

Posted on

April 1, 2013

Posted under

Bias, evidence based practice, Evidence Resources

Comments

Leave a comment

Why Can’t Guideline Developers Just Do Their Job Right????

I am reviewing a manuscript about the trustworthiness of guidelines for a prominent medical journal. I have written editorials on this topic in the past (http://jama.jamanetwork.com/article.aspx?articleid=183430 and http://archinte.jamanetwork.com/article.aspx?articleid=1384244). The authors of the paper I am reviewing reviewed the recommendations made by 3 separate medical societies on the use of a certain medication for patients with atrial fibrillation. The data on this drug can be summarized as follows: little benefit, much more harm. But as you would expect these specialists recommended its use in the same sentence as other safer and more proven therapies. They basically ignored the side effects and only focused on the minimal benefits.

Why do many guideline developers keep doing this? They just can’t seem to develop guidelines properly. Unfortunately their biased products have weight with insurers, the public, and the legal system. The reasons are complex but solvable. A main reason (in my opinion) is that they are stuck in their ways. Each society has its guideline machine and they churn them out the same way year after year. Why would they change? Who is holding them accountable? Certainly not journal editors. (As a side note: the journals that publish these guidelines are often owned by the same subspecialty societies that developed the guidelines. Hmmmm. No conflicts there.)

conflict of interest

The biggest problem though is conflicts of interest. There is intellectual COI. Monetary COI. Converting data to recommendations requires judgment and judgment involves values. Single specialty medical society guideline development panels involve the same types of doctors that have shared values. But I always wonder how much did the authors of these guidelines get from the drug companies? Are they so married to this drug that they don’t believe the data? Is it ignorance? Are they so intellectually dishonest that they only see benefits and can’t understand harm? I don’t think we will ever truly understand this process without having a proverbial fly on the wall present during guideline deliberations.

Until someone demands a better job of guideline development I still consider them opinion pieces or at best consensus statements. We need to quit placing so much weight on them in quality assessment especially when some guidelines, like these, recommend harmful treatment.