Advanced EBM Elective for Medical Students

I teach an advanced EBM co-enrolled elective in the Spring semester to 3rd and 4th-year medical students.Below is a screen shot of the course home page.

advanced-ebm-screenshot

This year I decided to revamp it completely and make it more philosophical. It was a challenge to decide what topics to include. You can see the topics I plan to cover along the right side of the image. I wanted them to be thought provoking and also useful at some level and not something taught in the usual EBM course.  I also made some other big changes after being inspired by 2 classes (Introduction to Openness and Social Network Learning) I recently took as part of my Master of Educational Technology degree (Thanks  Fred Baker and Jackie Gerstein). Here are the changes:

  • It’s open (all materials are free to use and anyone can take the course at any time). I hope to get some non-UAB students to take the course at the same time as my UAB students- a mini-MOOC so to speak.
  • Students will use social network tools extensively to enhance learning.
  • Students get a say in what they learn. I have designed the learning modules but I encourage students to develop their own learning module to replace one of the ones I developed.
  • Students will learn the value of social media for finding, creating, and sharing information.
  • Students will learn about personal learning networks and how to cultivate them.
  • Students will be exposed to learning activities they probably haven’t been exposed to in the past or used very much (e.g. jigsaw activity, curation,  concept mapping, blogging, tweeting) in medical education.

I hope these changes will enhance the ability of students to make meaning of this material. If nothing else I enjoyed creating a new class.

Workshop on Developing Open Educational Resources

On Friday September 23, 2016 I am presenting a workshop on developing open educational resources (OERs) at the UAB Research and Innovations in Medical Education conference.

This Hyperdoc is a self-guided version of the workshop.

These are the Google slides I will use at the presentation.

I became very interested in openness during recent coursework for my Master in Educational Technology degree. I blog about my experiences in that course here.

If you want a good overview of openness download The Battle for Open ebook by Martin Weller.

Now it’s your turn: Tell me what you think of the materials or open resources/learning/publishing in general.

Learning materials for “Make Your PowerPoints Evidence-Based” workshop

I did a workshop on how to design multimedia slides to be consistent with Mayer’s Cognitive Theory of Multimedia Learning. The workshop materials are below.

 

https://docs.google.com/presentation/d/1zNPsZNNODQxywAH9QZxi6mSgrxh4P08UYThO5ckfP5Q/edit?usp=sharing : These are the Google slides I used for the workshop.

 

Here are 2 handouts that I used:

1. Goals of Instructional Design Handout : reviews methods to reduce extrinsic cognitive load, manage intrinsic cognitive load, and foster germane cognitive load

2. Make Your PowerPoints Evidence-Based handout used during the workshop

Evidence-Based Teaching Principle 3: Modality Principle

The following is a slide I might use to begin teaching about p-values, type I and type 2 errors. What do you think about it? Will students learn deeply from it? (Would like to see a larger version of the slide? Please click on it)

Version 1

Version 1

Or do you think students would learn more deeply from this slide? The words at the bottom of the slide would be spoken by the instructor while the graphic is displayed.

Version 2

Version 2

Research would predict version 2 is better and will lead to deeper understanding. But why? What is different about them?

Version 1 violates the modality principle which states that people learn more deeply from multimedia lessons when words explaining concurrent graphics are presented as speech rather than as on-screen text. In version 1, the visual channel would have to simultaneously process the graphic and the printed text. This would likely overload this channel. In contrast, in version 2 the education message is split across separate cognitive channels- the graphic in the visual channel and words in the auditory channel.

Some caveats or limitations of this principle:

  1. It’s more important for novice learners
  2. It’s more important if the material is complex and presented at a rapid pace in a lecture. If the learner can control the pace of the material the modality principle is less important.
  3. Doesn’t apply if only printed words are presented on the screen (without any corresponding graphic)
  4. There are times when words should be presented on screen
    • words are technical
    • words are not in the learner’s native language
    • words are needed for future reference (e.g. directions to a practice exercise)

What’s the evidence for this? The modality principle is supported by more research than any other multimedia principle. Mayer identified 21 studies published through 2004 and found an average effect size on transfer tests of 0.97 (effect sizes > 0.8 are significant, 0.5 are moderate).

Evidence-Based Teaching Principle 2: Contiguity Principle

The following is a slide I might use to teach about interpreting a forest plot. What do you think about it? Will students learn deeply from it? (Would like to see a larger version of the slide? Please click on it)

Version 1

Version 1

Or do you think students would learn more deeply from this slide?

Version 2

Version 2

Research would predict version 2 is better and will lead to deeper understanding. But why? What is different about them?

Version 1 violates the spatial contiguity principle which states that people learn more deeply from a multimedia message when corresponding words and pictures are presented near rather than far from each other on the page or screen. In version 1 the words describing the image are at the bottom of the slide. The learner will have to look away from the graphic to find this description and then hold it in working memory (remember working memory is limited in capacity and time it can hold an object) while he looks back to the image and tries to process them together. This can overload cognitive capacity and impair learning. Version 2, on the other hand, has the words right next to the corresponding graphic thus reducing cognitive work. This is especially important when words refer to parts of on-screen graphics.

Other common violations of the spatial contiguity principle  include:

  • Feedback is displayed on a separate screen from the practice exercise or question
  • Directions to complete practice exercises are placed on a separate screen from the application screen
  • Key elements of a graphic are numbered but the legend is at the bottom of the screen

Watch the following video about how to calculate the number needed to treat. Will students learn deeply from this video?

Research would predict they won’t because the instructor violated the temporal contiguity principle which states that people learn more deeply from a multimedia message when corresponding animation and narration are presented simultaneously rather than successively. Cognitive capacity will be overloaded because the learner has to hold all of the relevant words in working memory until the animation is presented. This principle is especially important when narration and animation segments are long and when students can’t control the pace of the presentation.

What’s the evidence for this? Mayer, in Table 12.7 in the Cambridge Handbook of Multimedia Learning (2014), summarizes 22 studies on spatial contiguity published through 2012 and finds an average effect size of 1.10 (effect sizes > 0.8 are significant, 0.5 are moderate). Table 12.8 summarizes 9 studies on temporal contiguity published through 2008 and finds an average effect size of 1.22. Thus, following the contiguity principle leads to deeper understanding.

Evidence-based Teaching Principle 1: multimedia principle (Use words and pictures rather than words alone)

The following is a slide I might use to teach about one of the criteria for critically appraising a therapy study. What do you think about it? Will students learn deeply from it?

Version 1

Version 1

The multimedia principle states that people learn more deeply from words and pictures than from words alone. Why might this be? Reflect upon the cognitive theory of multimedia learning and think about why the multimedia principle leads to better learning.

Here is another version of the previous slide that better adheres to the multimedia principle. (Note: Would you like to enlarge the image? If so, please click on it). What do you think about this one? Will students learn more deeply from it or version 1?

Version 2

Version 2

Where are the words you say? They would be spoken during a lecture explaining the same information on the version 1 slide. They just aren’t typed out on the slide. Another format would be to put the written words in the notes area in PowerPoint.

Research would predict that students will learn more deeply from version 2 than version 1. Why? Remember active processing occurs where we take words and images and develop verbal and pictorial models. The words and images work together to help learners develop the models. Words alone can lead to more cognitive work for the learner to construct a model. Also, words alone might not be effective in activating prior knowledge which we need to do so that it can be integrated with our new model which then leads to learning. I have left out an important explanation here (can you guess what it is?) but it is a multimedia principle of its own and will be covered in an upcoming post.

Are all images created equal? What kind of images should I use? The answer to both of these question is that it depends. Lets focus on what you are trying to teach first. If you are trying to teach a motor skill or complicated manual tasks animated images or video seems to work better. Static images are better or just as effective as animation for everything else.  Static images seem to be better for promoting deep understanding.

Which graphic below do you think would lead to better understanding about heart function? A or B?

From Butcher. J of Educ Psychol 2006;98:182

From Butcher. J of Educ Psychol 2006;98:182

Butcher (2006) found that simpler visuals (a) led to better understanding. The simpler visual led learners to make more attempts to understand how the heart works than the complex visual. Making more attempts led to better mental models. It seems that too complex of images can overwhelm novice learners.

Some caveats or limitations of this principle:

  1. If learners can control the pace of instruction complex images promoted stronger knowledge gains (in a lecture setting where the instructor controls the pace simple images are better)
  2. Its more important for novice learners
  3. Sometimes only words can be used to explain a topic

What’s the evidence for this? Mayer, in Table 7.1 in the Cambridge Handbook of Multimedia Learning (2014), summarizes 9 studies published through 2006 and finds an average effect size on retention tests of 0.19 and for transfer tests of 1.63 (effect sizes > 0.8 are significant, 0.5 are moderate). Thus, this principle shows weaker effects for retention but good effects on deeper understanding.

Evidence-based teaching of EBM (and anything else)

I am going to have a series of posts on multimedia teaching principles. I am pursuing a master degree in instructional design and educational technology and as I am learning about instructional design I am realizing how poorly I have designed much of my teaching materials.  Furthermore, violations of the principles I will discuss in this series is very common in medical education.  Its not the fault of the instructors as they haven’t been taught these principles. For some reason in medicine we assume doctors and PhDs know how to teach.

The point of this series will be to present multimedia design concepts that have been proven in the educational literature to improve learning based on tests of retention (do you remember the content based on simple recall) and transfer (can you apply the information to solve a closely related problem).

We all teach using multimedia materials. If you put words (spoken or written) and images together in a presentation that is a multimedia presentation. So this series will be applicable to all teachers.

This first post will set the stage for future posts. The theory upon which all other posts will be based is the Cognitive Theory of Multimedia Learning by Richard Mayer.

cognitive theory of multimedia learning

The main components of this theory are as follows:

  1. Dual channels: there are 2 pathways to process information: auditory and visual ( designated in blue and green, respectively)
  2. There is limited capacity of each pathway to process information
  3. Active processing occurs in each pathway

When words or images are presented to us we first have to determine which words or images are important (or which portions of them are important).  After we select words, images, or sounds that are meaningful, we organize them in our working, active memory into verbal and/or pictorial models. We then actively integrate these models with activated prior knowledge to create new knowledge (learning).

Multimedia presentations should be designed to facilitate this process. During the remainder of this series I will present evidence-based ways to do this. I will delve further into the Cognitive Theory of Multimedia Learning when I discuss how this process is affected by each of the design principles.

Mix up your EBM teaching

Actually you should mix up all your teaching. Keep it fresh and interesting to your learners. I am getting a Master in Educational Technology and as part of one of my classes I had to design a WebQuest activity. You can learn more about WebQuests from WebQuest.org.

A WebQuest is an inquiry-oriented lesson format in which most or all the information that learners work with comes from the web.

I developed a WebQuest on determining pretest probability. You can look at my WebQuest here. It is geared toward 3rd and 4th year medical students but could be used with residents also. Feel free to use it.

probabilityheader2

While using the Web to teach is nothing new it is good to change up how you teach. Instead of lecturing on determining pretest probability have your learners do this WebQuest on their own time and flip that part of your lesson on diagnostic testing. It can also be done during class time and many of the WebQuests on WebQuest.org are designed for in-class use. While I had to actually develop/code the website there are online WebQuest creators (just look on the WebQuest.org site for the Creating WebQuests tab).

Journal Club- The UAB Experience

Just about every internal medicine residency program has a journal club.  One could argue about the evidence behind this activity but it seems to serve its purpose if nothing else than to make housestaff read some journal articles (and not just UpToDate!). I think it does serve a purpose of encouraging critical appraisal/thinking about research publications. Doctors will always have to read new research studies. It takes time for studies to be incorporated into secondary publications like Dynamed and UpToDate. Furthermore, not everything makes it into these evidence-based resources.  Also research (published in every journal) is full of biases that lead to departure of the findings from the truth. Critical appraisal is the only way to detect them.

journal club

This is not our flier but one I found on the internet that I thought was interesting

Since 1999 or so I have been intimately involved in the journal club at UAB. At times I have run it completely but now I serve more as a guide and EBM expert for one of the chief residents who puts it all together. I think it has gotten greater buy-in from the housestaff coming from the CMR instead of me.

So I thought I would cover some of what we have done at UAB. Not that we are the world’s beacon for journal club but we have tried alot of stuff over the years. Some of it failed….some of it successful.

Time of day: we have done everything from 8am, noon, to at night at a faculty member’s house. What has gotten the best turnout is 8am before their day gets started.

Article Selection: This has been a debatable topic since day 1. We have done several things:
1) Latest articles in major journals
2) Rotating subspecialty articles (one month cardiology, one month GI, etc)
3) Article chosen by resident based on problems they saw during patient care
4) Article chosen by me to prove an EBM principle
5) Now we seem to be focusing on articles written by UAB faculty so that they can come as an expert guest.
6) We are considering using classics in medicine articles that are the foundation of what we do (eg first article on ACE inhibitors in CHF) because current residents are unlikely to ever read these articles.

Format: We seem to vary this almost yearly:
1) Faculty reviews article and asks questions of the housestaff about what various things mean
2) Teams of residents argue for or against using a drug, etc against another team of residents
3) Each individual reads the article and comes to JC not knowing what they could potentially be asked
4) A handout with Users Guides questions and a few other questions on design or applying the information is given out ahead of time but is only discussed by those willing to answer
5) Same handout given but with individual residents assigned specific questions to answer (this was the first time we could show that the residents actually read the paper ahead of time)
6) Groups of residents work on questions outside of JC on their own time (usually 3rd yr resident assigned to coordinate the group meeting) with the expectation to teach the other groups at JC. (this worked pretty good actually)
7) Last year we went to a flipped learning format where I put alot of material on edmodo.com that the residents were to do ahead of time (if they needed to) with assigned questions to be answered by individual residents. They felt like this was too much work to go thru all the material online.
8) This year we are to perhaps our most successful format (from resident satisfaction standpoint) where a handout of questions is answered in JC as a group project. A faculty expert gives a very short didactic talk at 2 points during JC on a very specific EBM topic related to the article (eg what is a likelihood ratio). The only expectation is that the article is read prior to JC. We still use somewhat of a flipped format where I reference a short video or 2 to watch about topics in the chosen article but its much less time intensive than last year.

I think overall what has been successful for us is when JC has the following elements:
1) Group work. Engaged learning is always desirable.
2) Clinical and EBM faculty expert present. Seems to give the article a little more value.
3) Case-based. We always solve a real world problem. I always tell the CMR making up JC to make sure the residents walk away with something they can use clinically.
4) Flipped light– giving the residents some information, but not too much, that they can review about EBM principles leads to many of them actually watching the videos or reading background papers. They come much more prepared and have a good basic knowledge that we can then build upon.

Jazzing Up Journal Club

I am preparing to write something about what I think works well for journal club. We’ve tried lots of different things here at UAB. I would like to hear from you about what you do for your journal club. What works well in your journal club? What doesn’t work so well?

We need a dialogue here so please enter some comments. I’ll collate what I get and then give my own thoughts about Jazzing Up Journal Club.