Separating Signal and Noise: How IMPROVE-IT’s Modest Results Became a “Home Run”

Written by Stephen A. Martin, MD and Lisa Plymate, MD

Given a set of data, we’re used to separating signal from noise. In the case of the IMPROVE-IT study results initially released in November at the American Heart Association conference and published June 3rd in the New England Journal of Medicine, however, that signal is especially weak. But it’s been overwhelmed by corresponding noise in the media from study advocates.


How so? IMPROVE-IT [IMProved Reduction of Outcomes: Vytorin Efficacy International Trial], sponsored by Merck, enrolled only people with new-onset cardiovascular disease—within 10 days of hospital admission with high-risk acute coronary syndrome. The results over 7 years are:

a) An absolute decrease in myocardial infarction (MI) or stroke from 34.7% to 32.7% with no difference in mortality. The confidence interval, 0.89 to 0.99, was within 0.01 of crossing the null result
b) 49 out of 50 people (98%) treated with the ezetimibe intervention had no benefit (number needed to treat = 55)

Graphically, the results look like this:


In the actual paper, the Kaplan-Meier curve actually had to be magnified to see the difference between treatment groups!


Some observers found these results modest, small, and “very clinically marginal.” Though the study “technically” met a primary end point, that “just make[s] us want to look more closely” at the actual data. In essence, the signal is underwhelming.

So why do others see ezetimibe as a “truly spectacular result” and “home-run for patients”?


Because that’s the noise. Actually, at least 3 noises:

Broken Regulatory Structures, Research Models, and Clinical Application

The FDA’s reliance on surrogate markers for drug approval took an existential turn in 2008 when LDL lowering with ezetimibe provided no clinical benefit. Niacin, fibrates, CETP inhibitors, fish oil, post-menopausal hormone therapy, and dietary saturated fat have all failed in clinical outcomes despite their impact on lipid numbers. Hypertension, diabetes, and cancer treatments have similar problems. That’s why the Institute of Medicine and others (see here, here, here, here, and here) have pointed out the flaws of these surrogate shortcuts and called for regulators to be more cautious in their use—and weigh the risks of using them at all.

The complexities of regulation are complemented by deliberate strategies in research. In the case of IMPROVE-IT, a large sample size of highest-risk patients allows for statistical significance of a quintuplet end point driven by reductions in nonfatal outcomes. Interventions from these high-risk situations are inevitably promoted for unproven (but lucrative) low-risk primary prevention; most patients who now take ezetimibe are these low-risk patients. In addition, the hype now is that it isn’t only statins that are effective. Even though there has been no clinical trial comparing ezetimibe alone with ezetimibe in combination with a statin, ezetimibe (Zetia) is already being promoted as a substitute for statin-intolerant patients.

An additional hurdle is the oft-replicated gap between results obtained in highly-monitored trials and those in the clinical real world—generally referred to as efficacy versus effectiveness. If there is only a 2% absolute benefit in the enriched risk and monitoring in IMPROVE-IT, the likely outcome in an unselected population in the real world will be a much smaller number.

The Academic/Pharmacologic Complex

Concerns about this relationship have been well-established; IMPROVE-IT provides the latest case study. The general narrative involves 1) Academics who stake their reputation on a disease pathway that also coincides with a pharmaceutical company’s interests; 2) These academics then perform and promote studies funded by industry that benefit both themselves (as medical leaders) and the company (in terms of sales); 3) They also mention other studies that have no direct relationship with the research at hand to enhance their sciencey-ness, and 4) Subsequently, the academics act as thought leaders for practicing clinicians and help foreclose debate.

What does this look like in practice? A set of direct quotations from the week of IMPROVE-IT’s release provides the narrative:

IMPROVE-IT supports the “lower–is-better” cholesterol premise and provides proof of the LDL theory. People can stop yapping. In addition, a very large genetic study supports the fundamental truth that lower LDL means less CHD risk. If you’re developing another drug that cuts LDL cholesterol, this is great news. [D]octors … will now have a stronger belief in LDL.

Limiting dissent is a key part of this narrative. Which is why the AHA/ACC cholesterol guidelines introduced last year were unsettling to the LDL community. The guidelines treat cholesterol generally only with statins in a fixed-dose (similar to use of aspirin after a heart attack), unlike the preceding “target” strategy (LDL below a certain level). They walk away from lower and lower LDL.

Which is why the lead author and NEJM editorialists are quick to promulgate the idea of “lower LDL is always better.” Per Christopher Cannon, the lead author: “’This [result] would likely push the target, at least for these high-risk patients, even lower,’ he told MedPage Today, predicting that the guidelines would change.” The editorialists weigh in also, with the title “Proof That Lower Is Better—LDL Cholesterol and IMPROVE-IT.” In an editorial full of cognitive dissonance, the authors cite the null fibrate and niacin studies, then call this single ezetimibe study “proof.”

Follow the Money

Why all this noise over such a marginal result? Why the call for new guidelines—based only on this single non-peer reviewed pharma-funded trial? Because industry and its academic advocates need to claw back the guidelines to “lower is better.”

The inside baseball: If the guidelines aren’t changed, and if the FDA is more cautious about the limitations of surrogates like LDL, then new PCSK9 inhibitors, which also reduce LDL, may not be quickly approved and promulgated:

One another thing to watch for: if IMPROVE-IT results are positive … there is speculation that the FDA may favorably view the use of a surrogate marker – in this case, lowering LDL – when evaluating the PCSK9 inhibitors. These are a new type of injectable medicines that are being developed and a horse race is under way between Sanofi and Regeneron Pharmaceuticals, and Amgen.

This is why the Pharma chat rooms are abuzz. Writes one anonymous contributor on November 17th:

Why that means we can price our shiny new antibody at $1600 a year!! I’m so glad that the cost of making our shiny antibody is lower than the cost of making a statin. Besides everyone knows – most patients hate pills – they love injections !!!!!!

With the June NEJM release of the study, this connection was all the more evident. Noted one cardiologist: “Indirectly, but more importantly, these findings are likely to increase cardiologist and patient enthusiasm for the more potent PCSK9 inhibitors as the preferred adjunctive agents to high-dose statin therapy.” If “enthusiasm” is the coin of the medication realm, we are all in trouble. And if industry-funded studies are the main source of this enthusiasm, then let us be wary.

What’s Next?

Separating signal from noise is part of our job. In the case of ezetimibe and its future cousins, the signal is currently weak and the noise is strong. We need help from the medical, journalist, and regulatory communities to help us listen as best we can and make sure the public isn’t being sold a noisy tune.

Related Reading:

FDA Weighs Target Population for Amgen Cholesterol Drug