In this episode, I’ll discuss whether results from trials that were stopped early are trustworthy.
The short answer to this question is: It depends on why the trial was stopped early.
Perhaps the biggest pitfall is in using evidence from a trial stopped early for benefit. It has been reported 30 years ago that stopping trials early for benefit frequently leads to an overestimation of the size of the treatment benefit. The trouble is, stopping a trial early for benefit makes for a nice press release and news story: This medication is so good we were ethically obligated to stop the trial so the control group can also have access to it.”
There are many high-profile examples of a trial stopped early for benefit where subsequent studies find that the benefit is less pronounced, non-existent, or even that there is harm with the treatment.
Xigris (Activated protein C) was marketed as the first non-antibiotic treatment to improve sepsis mortality. It’s pivotal trial was stopped early in 2001 for an apparent benefit over placebo. This trial was published to much fanfare in New England Journal of Medicine. The medication found its way into sepsis guidelines and was even given a higher level of evidence weighting than antibiotics themselves. Subsequent trials identified bleeding risk and questioned the mortality benefit which ultimately led to the removal of the medication from the market.
Another example, also from 2001, is that of intensive insulin therapy in critically ill patients. A single-center randomized trial was stopped early and published with a finding of a large 40+% drop in mortality compared to the control group. The recommendation for intensive insulin therapy was widely disseminated throughout guideline recommendations. Years later, additional studies tempered the expectation for a mortality benefit and highlighted the risks of hypoglycemia with intensive therapy.
Yet another example is that of beta-blockers in non-cardiac surgery patients. After barely 100 patients were enrolled, this trial was stopped early and a 91% reduction in relative risk of mortality was claimed with bisoprolol. Subsequent trials and meta-analysis with over 10000 patients found that the mortality reduction is more likely an mortality increase, and a reduction in non-fatal MI is countered by an increase in stroke.
Based on these examples, trials stopped early for benefit should be treated with skepticism. It is possible however that a trial stopped early for benefit after enough events have occurred and with a p value <0.001 might be valid.
Trials stopped early for harm or futility are less problematic as there is no incentive to use the fact that a trial was stopped early for harm or futility to promote a medication.
Beyond stopping for benefit, harm, or futility, a 4th category is trials that were stopped for administrative reasons.
If a trial was stopped early because funding ran out or was withdrawn, the results may be trustworthy, provided that enough patients experienced the event being studied.
If a trial was stopped early because of low enrollment, I am willing to consider the results but I pay extra attention to the inclusion/exclusion criteria. Trials stopped early for this reason often have unrealistic inclusion/exclusion criteria that make applying the results to real-world practice problematic.
To get access to my free download area with 20 different resources to help hospital pharmacists go to pharmacyjoe.com/free.
If you like this post, check out my book – A Pharmacist’s Guide to Inpatient Medical Emergencies: How to respond to code blue, rapid response calls, and other medical emergencies.
Leave a Reply