Evidence-based Attire Part 1

I am going to write a 2 part series on what we should wear as physicians. I was inspired to do this when my resident said he has received an email declaring all male physicians must wear ties. I mostly have abandoned tie wearing as they are hot, uncomfortable and I just can’t find matching ties to many of the shirts I want to wear and don’t want to commit a fashion faux pas. I also perceived them to be an infection risk (I will address this in part 2 of this series). When I don’t wear a tie I wear my white coat. When I wear a tie I often forgo the white coat. What do patients think about this?

I felt it important to try to review a smattering of the literature on the topic. I admit I didn’t do a full literature review because the importance of this is just not worth the time. I did randomly select from the studies I did find so as not to only support my biased opinion that patients don’t really care what we wear.

There is a fair amount of literature on this topic in both the medical and dental world. Many of the studies included pictures like the following for patients to critique and give their opinion of the physicians depicted. They were often asked about the professionalism of the attire and their desirability of certain aspects of dress.

I didn’t find a dress option I really had hoped to find information on: white coat but no tie.

What do patient’s think about how we dress?

Below are some selected figures from studies showing these results. This first table shows that wearing a tie was fairly desirable but not as desirable as a name tag and white coat.

This is from a study of anesthesiologists (https://goo.gl/kTKUoM)

The figure below was from a study of anesthesiologist. It included 110 patients. You can see that wearing a tie was felt to be important by about 25-30% of them. Wearing a name tag and white coat was more important.

Another from a study of anesthesiologists (https://goo.gl/5fMh7k)

The figure below suggests that wearing a ties led to greater patient comfort in dealing with the doctor. Interestingly, not wearing a tie has the same ranking as wearing a white coat in patient’s comfort of dealing with a doctor dressed in such a way.

Finally, the table below shows patients in this study mostly disagreed that ties were important and didn’t even feel white coats were all that important. They felt is was most important for doctors to act professionally.

So, there you have it. White coats and name tags seem to be the most important. It helps patients identify us as doctors and we are perceived to be more professional in the white coat. Ties have mixed results. I am still undecided as to how I feel about this. I used to love wearing ties…now not so much. I think I will make my decision after I review the infection risk data. Stay tuned for that installment which will be coming soon.

What do you think?

Academic promotion and salary structure is part of the problem with research

There has been an interesting string on a listserv I am a member of about EBM being highjacked and the poor quality of research and of how so much research goes unpublished.  Below are two responses that got me thinking about something:

I am not impressed by the recent declaration in BMJ just because it is about the bad studies and the need of new EBM. What new? The authors of the declaration published calls for new EBM almost annually.  And most the time the calls are to go beyond evidence, and to do good, not bad.

Yes, too many research are poorly designed and executed. Is it new finding? Was ever EBM quiet about the bad evidence?

“today’s world …so financially driven” – was it ever not like that? Can one name the Golden Age?

If not kill your self, one need to live in the real world and try to make it better. EBM is specifically about it: critical appraisal and education (in wide sense). It is about use of the best available evidence for the good of the patients. Best available, sic. To influence research, to improve research, to clean the publication practice is a good thing, but it is beyond EBM in pure sense.

and…

From the link, ” Too many research studies are poorly designed or executed. Too much of the resulting research evidence is withheld or disseminated piecemeal”. In short, medical research is not implemented correctly for a number of reasons. So my question is should we scrap medical research? OR RCT’s which are supposed to be the gold standard for trials should be dropped from research?

For me the most logical answer is better implementation. So the debate should how we can do this in today’s world which is so financially driven. In other words, the debate/discussions should not be about research or RCT’s are flawed and hence should be replaced.

Also, please don’t blame EBM for shoddy research practices. EBM has no control on what researchers like to do or don’t like to do. EBM can only make recommendations on how to do better research and interpret research.

My miniepiphany why reading this string was that academic medicine is partly to blame for some of the problems in the evidence base. In academics you have to garner grants to pay for yourself (or at least part of your salary) and you have to publish to get promoted. So we have a system that rewards quantity over quality. Lots of research is done because we have to do it to get promoted. But how much of it is worthwhile research? Not much. Many academics aren’t trained in proper research technique so they perform lower quality studies.

So one fix to “EBM” could be to restructure how we finance academics. We need to quit focusing on grants (especially considering governmental funding of these goes down every year) for compensation. Teaching (and yes medical schools need to actually pay adequately for the teaching we do) and clinical activities should be the primary funders of academics. Then research could be done on important things and by properly trained researchers.

 

What can we do to reduce the amount of research that goes unpublished?

An excellent piece was published by Paul Glasziou and Iain Chalmers about the large percentage of research that goes unpublished. As they note, the estimate that 50% goes unpublished is likely an underestimate. Unfortunately they didn’t offer any significant solutions other than we need a “better understanding of the causes of, and cures for, non-publication.”

A “simple” solution is for the drug/device approval process to require all studies related to that product and/or conducted/funded by the requesting company be registered and published. This would miss studies done after drug/device approval or done by independent parties but a large number of nonpublished studies are conducted or funded by the companies that market the drug/device. This would also miss all the other studies not directly related to drug/devices (e.g. epidemiological studies).

Another significant challenge is where to publish this information. The web makes the most sense as this is the cheapest route of publication. Maybe the FDA (or some international commission) could have a page(s) on each drug that includes full text access to all studies done on that drug/device. Would these need peer and editorial review? Yes, but a daunting task as we already struggle to find willing and competent peer reviewers. FDA budgets shrink repeatedly and this would be a significant financial burden.

What I really wanted to do in this post was to give my thoughts on a  question raised by Jon Brassey (Director of the TRIP Database):

  • What is better a large RCT or a SR based on a “biased subsample”?

Is a large RCT more desirable than a systematic review (SR) based on a biased subsample of studies? This has been a conundrum for some time. You can argue both sides of this. The reason he says biased subsample is that we know more positive studies get published than negative, larger effects get published more than small effects, etc. Is the answer to this question “it depends”? It depends on your goals: a more precise estimate of the biased effect (favors SR), more generalizability (favors SR), a potentially more methodologically sound result (favors RCT). What is interesting to consider is that the same study repeated over and over will result in a distribution of results (this is why it shouldn’t surprise us that when we do seemingly the same study we don’t get the exact same result). Should we repeat studies? When should we stop repeating the studies (i.e. when have we adequately defined the distribution of results)?

I don’t think we can really answer this question as both of these study types have limitations but if I had to pick one I would rather have a large RCT that is well done than a SR based on a limited subset of the data especially considering we don’t know what is missing and the effect seen in those missing studies.

 

Guidelines should not include opinions

The authors of this viewpoint have it wrong on a couple of fronts.

“The purpose of practice guidelines must be to develop the best possible recommendations from a body of evidence that may be contradictory or inadequate.”

While I agree that having recommendations come from an expert body is useful when there is inadequate or contradictory evidence I don’t think they should be labeled guidelines. A consensus statement is a more appropriate term. After all, if evidence is lacking or contradictory aren’t these experts just giving their opinion? Isn’t it possible that another group of experts would give a different opinion?

So don’t label it a guideline. That term has garnered reverence that was never intended. Guidelines become law almost. They are bastardized into punishing performance measures and become the cornerstone of legal argument. So, the term guideline should not be used lightly.

“…but those recommendations should always represent the best evidence and the best expert opinion currently available.”

NO! No expert opinion. Data is too open to interpretation. Humans filter information using prior knowledge, experience, and many heuristics (including, very importantly, the affect heuristic). A person’s specialty really influences how they interpret data. It’s one of the reasons it’s so important to have multidisciplinary panels so that conflicts and heuristics can be balanced. Unfortunately, most guideline panels are very homogeneous and conflicted.

I agree that we need unambiguous language in guidelines. They should only contain recommendations on things that have strong evidence that no one refutes. When they venture into the world of vagaries they become nothing more than opinion pieces.

What say you?

If no one can give an understandable definition of p-value why do we use it? Or quit saying statistically significant.

In this post I might come across as sounding dumb… but it won’t be the first or last time that will happen. I came across this article about how scientists cant even explain p-values in an understandable way. I know the students and residents I teach only understand it as indicating statistical significance.  It then occurred to me as I thought of the definition that maybe we should get rid of p-values altogether and let users decide if what they are seeing is clinically significant.

What is a p-value?

First, we have to understand one other concept- the null hypothesis. When doing a study researchers have to construct a null hypothesis that they will attempt to prove or disprove (depending on how you look at it) with their study. For a superiority trial, the null hypothesis is that there is no difference between the intervention and control treatments. For a equivalence trial, the null hypothesis is that there is a difference between the intervention and control treatments. Data from each arm of the study are compared with an appropriate statistical test yielding a test statistic (e.g. t statistic if using a t-test). That test statistic is compared to a table of critical values for that test statistic to determine if the value (of your calculated test statistic) is greater than the critical value required to reject the null hypothesis at the chosen alpha level.  If it is, you reject the null hypothesis and say it is statistically significant. If not, you fail to reject the null hypothesis and the finding is not statistically significant.

P-values are simply probabilities. They are the probability of finding what was found  in the study (or even a bigger finding) if the null hypothesis was true.

Here’s an example. Let’s say I conduct a study to examine the effect of reading for 30 min each day on medical certification examination scores. My null hypothesis is that reading will not improve exam scores.  I randomize one group of participants to read for 30 min every day and the control group to no reading. Both groups take the certifying examination and I calculate mean scores for each group. I can compare these means with a t-test (assuming what parametric tests require is met) and I find the mean examination score is 5 points higher in those who read for 30 minutes compared to those who didn’t with a p-value of 0.04. So, this  p-value of 0.04 means that there is a 4% probability that you would see at least a 5 point higher mean score in the reading group given that there is no effect of reading on exam scores. What if the p-value was 0.2? Then there is a 20% probability that you would see at least a 5 point higher mean score in the reading group given there is no effect of reading.

A common mistake pointed out by statisticians is for someone to interpret the p-value as the probability of what was seen in the study being due to chance alone. I think many people think of it this way because it’s easy to comprehend. But chance isn’t the only explanation for a false positive finding in a study.

All this is very confusing, right? Exactly. So, if no one really understands p-values is it time to abandon them and the concept of statistical significance? After all, the 0.05 cutoff is just a tradition. Is 0.07 all that different? Or 0.051?

I know if I am suggesting we get rid of the p-value I have to suggest an alternative. The only one I can think of are confidence intervals but the statistical definition of that is confusing and not clinically useful. So, should we abandon confidence intervals too? Both the p-value and the confidence interval give useful information but if that information is interpreted incorrectly can it really be useful information?

What should be the role of the p-value and the confidence interval? Maybe we just need to better educate users of the scientific literature on what these values can and cannot tell us then all of this would be moot.

What do you think?

Advanced EBM Elective for Medical Students

I teach an advanced EBM co-enrolled elective in the Spring semester to 3rd and 4th-year medical students.Below is a screen shot of the course home page.

advanced-ebm-screenshot

This year I decided to revamp it completely and make it more philosophical. It was a challenge to decide what topics to include. You can see the topics I plan to cover along the right side of the image. I wanted them to be thought provoking and also useful at some level and not something taught in the usual EBM course.  I also made some other big changes after being inspired by 2 classes (Introduction to Openness and Social Network Learning) I recently took as part of my Master of Educational Technology degree (Thanks  Fred Baker and Jackie Gerstein). Here are the changes:

  • It’s open (all materials are free to use and anyone can take the course at any time). I hope to get some non-UAB students to take the course at the same time as my UAB students- a mini-MOOC so to speak.
  • Students will use social network tools extensively to enhance learning.
  • Students get a say in what they learn. I have designed the learning modules but I encourage students to develop their own learning module to replace one of the ones I developed.
  • Students will learn the value of social media for finding, creating, and sharing information.
  • Students will learn about personal learning networks and how to cultivate them.
  • Students will be exposed to learning activities they probably haven’t been exposed to in the past or used very much (e.g. jigsaw activity, curation,  concept mapping, blogging, tweeting) in medical education.

I hope these changes will enhance the ability of students to make meaning of this material. If nothing else I enjoyed creating a new class.