I received an email about an in introductory video I made for medical students about intention-to-treat (ITT) analysis. The author of the email also made a comment about the video on YouTube. The points he makes are valid and he expands on them in this manuscript. I will defend, if you will, the concept of ITT analysis.

First, ITT analysis refers to analyzing all patients based on the arm of a study they are randomized to no matter what happens to them…”Once randomized, always analyzed”. Even if they don’t take the intervention or become non-compliant their outcomes count against the group to which they were randomized. It is considered the primary analysis for superiority studies (where we want to prove an intervention is better than something else).

Important background disclaimer: ITT analysis is just one part of the methodology of a randomized controlled trial. (The other parts being concealed allocation, randomization, blinding, equal cointerventions, etc.) There is a tendency of critics of ITT analysis to think of it in isolation and I think that is a mistake. It is just part of an overall plan for conducting a study.

Randomization is considered the gold standard methodology for allocation in therapy studies. It is done for a couple of main reasons: 1) to equalize both known and unknown prognostic or confounding factors and 2) to prevent selection bias or cherry picking of patients to one arm of the study or another based on prognosis. In general, ITT analysis helps preserve the effect of randomization. If you start pulling out people differentially from each arm of a study the remaining people (to be analyzed) are likely not balanced prognostically. Yes, you can adjust for disparities between groups but only those that are measured. You can not adjust for unmeasured confounders. Randomization balances even unmeasured counfounders (assuming sample size is large enough). So its important to preserve randomization.

Dr. Feinman points out in his comment that “In the experiment, people sensibly want to know what is the effect of surgery compared to aspirin. Instead, ITT answers instead what is the effect of TELLING PEOPLE to have surgery vs. taking aspirin. Is that really what you want to know?” This is true but this is what we do in practice. I recommend a strategy to a patient and they will either follow it or they wont. So maybe it’s not the ITT analysis that is the problem but the question of the study. Another piece to this is that a per-protocol analysis can (and probably should) be done after the ITT analysis to see the effect in those who actually got the therapy. The results of both analyses should be reported so the reader can better understand the results. Data on dropout/noncompliant patient demographics/comorbidities/etc needs to also be reported so we can better understand potential reasons for drop outs or noncomplaince.

I mention in the video that ITT analysis is more conservative in that it will  less likely lead you to falsely reject the null hypothesis and falsely conclude that the intervention is effective. This is because those in the intervention group that dont get the intervention are actually like the control group; leading to more similar event rates and more difficulty in finding a significant difference.  So using ITT analysis somewhat depends on the risk to the patient of falsely saying the intervention is effective (when it isnt) or falsely saying it isnt effective (when it is). Conversely, per protocol analysis is less conservative in superiority studies.

I’m not sure what Ive written here will change anyones mind. I think a compromise is to use both ITT analysis and per protocol analysis and report both results. Hopefully, they are similar or at least qualitatively similar.

What do you say about this controversy?