A spat has broken out among researchers over the extent to which the controversial Chantix quit-smoking pill is linked to cardiovascular risks. A meta-analysis that was published last week in BMJ found the drug, which is sold by Pfizer, does not increase the risk of heart attacks and strokes. The results contrast with a meta-analysis published last year that maintained Chantix does increase heart risks, a conclusion the authors of the latest study labeled "misleading."
The dispute centers on varying approaches to pursuing a meta-analysis which, of course, combines the results of several studies to test a hypothesis. Beyond disagreements over methodology, however, the clash is certain to extend the wider safety debate over Chantix, which has also been blamed for suicidal and hostile behaviors, although the FDA maintains that studies show the benefits outweigh the risks (see here and here).
Safety concerns, you may recall, have plagued Chantix almost since the pill was approved in 2006 and, consequently, have frustrated Pfizer, which had high hopes the drug would generate impressive sales. Instead, a stream of media stories about psychiatric and cardiovascular side effects have dampened expectations. In the first quarter of this year, Chantix generated just $178 million, a 10.5 percent drop from a year earlier (see page 36 here).
The cardiovascular concerns were highlighted last year when the FDA added a warning on the product labeling about an association with a small, but increased risk of cardiovascular adverse events in patients with cardiovascular disease (read here). But the issue accelerated when a meta-analysis published in the Canadian Medical Association Journal found Chantix was associated with a 72 percent increased risk of serious adverse cardiovascular risks in smokers without a history of heart disease (look here).
The results of that study, which examined 8,216 patients in 14 trials, is what prompted researchers at the University of California, San Francisco, to conduct their own meta-analysis. In their view, the numbers simply did not add up. And when they finished reviewing 22 trials with 9,232 patients, they found a difference in risk of serious cardiovascular events was 0.27 percent between those on Chantix or a placebo, which was not clinically or statistically significant (here is the BMJ study).
"What caught my eye was that, in their very first study, they found one (CV) event in 400 people and the placebo was 0 out of 200," says lead author Judith Prochaska, an associate psychiatry professor at the University of California, San Francisco, and a researcher with the Center for Tobacco Control Research and Education. She notes she has also won an “investigator-initiated research award” from Pfizer that has been applied to this Chantix trial.
"There were twice as many subjects on Chantix and there was a difference of only one event. The calculated statistics for the study was 4.5, or 4.5 times greater risk and that's alarming. But the statistic they were using was unstable and inflates findings under certain conditions. And there were eight studies (in the CMAJ analysis) that were like that," she says, adding that there were other methodological issues that account for the different findings.
For instance, the CMAJ analysis excluded eight trials with nearly 1,600 tobacco users who were randomized to Chantix or placebo that did not have any serious cardiovascular events. And the CMAJ analysis examined events that occurred during the complete length of the Chantix trials, some of which lasted up to a year. Prochaska examined events only up to 30 days after a patient stopped using the pill in the belief the drug remains in the body for seven days after usage ends. And 13 of the 14 studies in the CMAJ analysis experienced greater attrition in the placebo group than in the test group, which she says could inflate the treatment effect.
The lead author of the CMAJ meta-analysis, however, contends the latest meta-analysis is off base. "I think there should be scientific debate about the best methodology,"Sonal Singh, an assistant professor of medicine at Johns Hopkings University, tells us. "I don’t think their study is misleading. There are just two different approaches to looking at the data. But I’m disappointed in the tone. But they needed media attention" (here is the UCSF press release that uses the word 'misleading' to describe his meta-analysis).
Singh noted that his meta-analysis examined patients through the duration of the Chantix trials because what remains unknown is the the length of time that a heart risk may appear after treatment ends. "We were learning from the Vioxx issues. With Vioxx, heart risks didn’t climb until long after people were taken off the drug. She assumes the potential CV risk is due to direct effect of the drug being in the body, so when the drug is out the risk should go away. I’m making a very different assumption. I don’t know how the drug is increasing heart risk, so I’ll count data through the end of a study and some lasted up to a year," he says. "Which world do you live in? She’s assuming she knows how the drug causes CV risks and I’m assuming I don’t know. And those are the kinds of studies that are needed."
In a response to BMJ, Singh and a colleague who worked on the CMAJ meta-analysis, Yoon Loke of University of East Anglia in the UK, write the journal that the review conducted by the UCSF researchers "has limitations in data, analysis and interpretation which raises doubts about the veracity of their conclusions." And they cites these points...
Prochuska and her colleague excluded a number of cardiovascular adverse events from the Chantix arms of the clinical trials that have been reported. He also says they analyzed data by treatment level, which would allow exclusion of events occurring in randomized patients. "In contrast, we adhered to the intention-to-treat (ITT) analysis in accordance with FDA regulations and established and generally accepted scientific principles," they write.
From there, they say the UCSF researchers "recommend risk difference as the most appropriate model." However, he maintains "most regulatory meta-analysis of safety risks, including our meta-analysis, are conducted on the relative scale." And they claims the models used by Prochuska "are statistically underpowered at low event rates, and bias their estimates towards the null" hypothesis. Finally, they say there was a failure to provide info on the optimal information size or the power of the meta-analysis (here is his letter to BMJ).
"There is a simple explanation why this study could not detect a difference in cardiovascular risk. Because of a weak design it was unable to detect any effect on anything," Thomas Moore, a senior scientist with the Institute for Safe Medication Practices and who serves as a consulting expert in the civil litigation regarding Chantix, writes us in an e-mail. "It would be a serious scientific error to make a safety claim based on a study that did not disprove the null hypothesis." He also co-authored a study with Singh that concluded Chantix is not suitable for first-line use (see this).
As for Prochaska, she dismisses the criticism as well as concerns that her findings were colored by her support from Pfizer. She maintains that she did began the meta-analysis before starting the research sponsored by Pfizer, which takes place in a hospital setting, because she did not want to put patients at risk. "It's unfortunate that anyone would throw away all the math we've done" for that reason, she says.
"This is the first time I've ever taken Pfizer funds. And this study is an idea I put forward. I have five other grants funded by NIH, the state of California. I don't feel any need that I have to take money from pharm. There are a lot of top scientists who do partnerships with pharma. It's not always sinister. And Pfizer had no oversight of this. There was no prior publication review and it was a completely independent analysis. It comes down to the math. We used the same method as Singh to identify studies and code events, and this shows in a very straightforward way why the statistics they chose were inappropriate."
43 Comments
http://www.ncbi.nlm.nih.gov/pubmed?term=singh-s%20moore-thomas
Thanks for the note. I actually have a link to the item I ran last November about that study, and a link to that study is embedded in that item. However, I should have gone a step further and noted he and Singh were co-authors. So I have added those few words and hope that it is now clear.
Appreciate the assist. Ed
Quelle surprise . . .
But as I have noted many times, conflicts of interest can have many sources. The 98% of papers listed in Retraction Watch that had no industry funding supports the idea that academic careerism and the eternal struggle to get grants is sufficient COI to induce outright fabrication of data, let alone bias. I don't think the issue is as simple as suggesting that that one of these authors has a COI, and the other does not.
What I find telling is that Dr. Singh wrote an article that concluded that very widely used drug confers a high level of scientific risk, and in spite of the importance of the subject matter, was unable to get this paper published in a first tier journal. This suggests to me that his analysis was rejected by the referees at higher quality journals such as BMJ.
Thanks for your post. I believe that you have hit upon important points in this debate.
If a pharmaceutical company did that sort of slight of hand in promoting a drug, they would become a topic of one of Ed's columns.
Also, while realizing that it is just a formalism, a 95% C.I. that includes 1.0 is not statistically significant. At least that's the convention that the FDA holds companies to that are trying to get drugs approved.
I was a closet smoker 1-3 cigs. per day. Grandchildren would be DEVASTATED if they knew NaNa smoked...so, it was time. PAST time.
Took two scripts back/back in '07. Was THRILLED when smoking was no longer pleasurable!! Holy smokes!!! Is it THIS easy to QUIT???!!!! Why didn't someone invent Chantix YEARS ago??!!!! Couldn't beLIEVE my luck!!!!!!!!!
And, had it ENDED there, I, TOO, would be singing Chantix' praises!! But, it DIDN'T. This is one, SNEAKY, POWERFUL, unPREDICTABLE drug!!! Once in your system, only God knows how and where you will land. Never in my wildest nightmare...would I have believed a little, blue pill could wreak such HAVOC!!!! But, it did...my life and world as I knew it...came to an end. My life was turned upside down and INSIDE OUT!!!
My dreams were no longer my own. Dreamt of deceased, loved ones coming to me at nite, HAPPY, HEALTHY, and BEGGNG me to come w/them!! Tried DESPERATELY to assure me my JOB WAS DONE HERE!!! My job which I've loved for 23 years and my husband of 37 yrs. would certainly have no problem replacing me. My daughters were busy raising their OWN children...wasn't sad, mad, glad...just flat. Totally flat and EXHAUSTED.
It seemed FUNNY to me to STILL be here... TAKING UP SPACE!! (That was my distorted/TWISTED thinking at that time! I was so tired of being confused...and TIRED!!!
Two hospital stays in '08 PLUS a psyche unit stay ROCKED ME TO THE CORE!!!! The stays were for: (1) severe chest pains (couldn't beLIEVE a person could FEEL such pain and not PASS out!!) Kept a week while receiving extensive testing from top to bottom, intraveneous fluids ad antibiotics, oxygen and morphine. Released w/diagnosis - - possible virus. (2) Three months later, drove car to rural area, sang to oldies on the way...picked up three bottles of sleeping pills and razors along the way.
Woke up six days later when being taken off a ventilator...surrounded by devastated, confused, family members. Do I even need to go into detail on the grovelling HELL its been for the past 4 years???!!! My trust in my doctor and pharma is destroyed/DISSOLVED. No longer do I trust ANYone involved in getting a new drug to market. Never before did I have a REASON to dig and look behind closed doors of big pharma, FDA, Doctors association w/both...etc. o...m....g. What a cold, GREEDY world we live in. I feel so stupid....naive...FOOLISH...for believing BLINDLY all of the above.
You almost GOT me Pfizer!!...and had I not survived, which is not a stretch...no one...but NO ONE...would've put Chantix in the equation!! Hell, I didn't even put it in there mySELF...it took an outsider to point out the possibility of the association!!!! I could've been 10 toes up...and my family would've chalked me up to NUTS!!!
I don't mean to discount your experience in any way. But I think it is fair to point out that lots of people do quit smoking with the aid of Chantix and do not experience side effects similar to those you did.
And by and large, those people with a more successful experience don't show up on message threads like this one to tell the other side of the story: the surgery, pain, and early cancer death they did not experience because they successfully quit smoking.
Addictions, whether smoking, alcohol, drugs, or others are very hard to break. Wouldn't it be nice if we could all just decide to no longer be addicted? It is not realistic.
Quite frankly, the Church of Cold Turkey is arrogant, and they don't care or sympathize with those who suffer from addiction. The media loves the cold turkey advocates from Harvard and Johns Hopkins because they are a fantastic foil to the "evil" pharmaceutical industry.
Chantix is one of many options that can help people quit. The more options the better.
I don't believe my addiction is physical. I have one before work...one/two AFTER work...none at nite...and none on weekends cause I'm busy babysitting my grandkids!! This is how I've been smoking for years.
When Chantix hit the market...I hit the bricks. Was beYOND excited to get started!! Thought for SURE I'd kick it w/THIS!! I WANTED it to work. And, it DID. For awhile! Then, when I started reaching again, filled another script. THAT'S when my problems began. Something went TERRIBLY wrong this time around. Yes, it DID touch on the smoking receptors...which, was GREAT...but, it didn't STOP there. If it HAD, I, too, would be thanking Pfizer for their miracle pill!!
But, it didn't...and I can't. It was a long, slow process...yet very thorough. My dreams were so real...so vivid...so EVIL...I was so confused...and tired...it was getting difficult to decipher my state! I'm not lying, nor exaggerating. This little, blue pill OWNED me. I know I didn't question or doubt for a minute the people involved in getting this script filled. I TRUSTED my doctor, my pharmacist, big pharma, and MOST OF ALL, the FDA!!! Found out the hard way what a gullible, naive, FOOL I am!!
They HURT me! I paid dearly.. physically and psychologically! Ties have been irrtievably/PERMANENTLY broken/severed over the trauma that was brought on my family. No one can give me back my quiet, comfortable, respectable, position as matriarch of my family. Too many people were permanently scarred watching a vent breath for me for six day as I lay in a coma. There is no going back. What's done is done...and I'm extremely forunate and HAPPY to be here...but let me assure any doubters/naysayers out there...no one has your back...you are a number...and the sooner we all absorb that the better. Cold, GREEDY, CORRUPT, world we live in.
You can't make this stuff up.
In their meta-analytical investigation of rare adverse events associated with varenicline use for tobacco cessation, Prochaska and Hilton compared four different methods of meta-analysis (the Peto odds ratio, and the Mantel-Haenszel odds ratio, risk ratio and risk difference methods) and noted that they yielded different estimates of effect [1]. Such differences are not unexpected since the methods are all based on large sample asymptotic statistical theory, the assumptions of which are challenged when events are rare. The three Mantel-Haenszel methods also involve the use of an arbitrary numerical correction to avoid computational errors that occur when attempting to divide by zero. Prochaska and Hilton conclude “for rare outcomes, summary estimates based on absolute effects are recommended and estimates based on the Peto odds ratio should be avoided”. They provide five arguments to support this statement, none of which is convincing, some of which are misleading, and some of which are seriously flawed. To demonstrate that a result is biased, it is necessary to know what the correct result should be. This cannot be achieved in a case-study such as this, which simply compares four different analytical methods. The authors’ conclusion is not justified and is potentially dangerous. First, Prochaska and Hilton state “treatment effects based on relative risks always are as or less extreme than those based on odds ratios”. It is a mistake to compare the ‘extremeness’ of two different metrics in this way. Furthermore, when the outcomes are rare as they are in this example, odds ratios are in fact very close approximations to relative risks. Second they argue “relative statistics cannot be calculated for trials with zero events and therefore can bias summaries against the null hypothesis of no effect”. This reflects a common intuitive feeling that it is wrong to exclude any studies from a meta-analysis. However, if no events occur at all in a trial, the trial in itself conveys no information about the relative odds or risks of events between the two groups. A meta-analysis may be viewed as a weighted average of trial results, with weights reflecting the amount of information each study contains about the summary statistic. Allocating a trial with no information a zero weight is entirely appropriate and does not introduce bias. Third, they argue that relative statistics hide the impact of the effect, whereas risk differences most clearly convey the effect, using this to justify their meta-analysis of risk differences. We agree that absolute effect measures convey more useful information. However, the use of relative measures in a meta-analysis is not a barrier to re-expressing the treatment effect in absolute terms, e.g. as a number needed to treat to benefit or harm, or a risk difference. Prochaska and Hilton also cite Vandermeer and colleagues’ analysis of 1613 meta-analyses [2], claiming that they showed the Peto odds ratio to be particularly biased. Vandermeer in fact compared results of asymptotic methods with meta-analytical techniques based on ‘permutation’ or ‘exact’ methods. As with the current study, Vandermeer did not know what the true result was, so was unable to indicate that one method was biased, only that methods gave different results. Finally they imply that the Peto odds ratio method must be biased because it produces the most extreme values of odds ratios for the individual studies. This is a misunderstanding of the way in which the method should be applied – it is designed to compute an overall meta-analytical statistic and not for computation of estimates of odds ratios for individual studies. The strength of the Peto method is that it aggregates within-trial comparisons across trials in a way that avoids the need for the arbitrary addition of 0.5 events to some treatment groups in which no events were observed. It is this arbitrary addition that causes both the Mantel-Haenszel risk ratio and odds ratio to be similar, and the Mantel-Haenszel odds ratio to be smaller than the Peto odds ratio. The rigorous approach to understanding bias in statistical methods is to undertake simulation studies, where the investigators create data with known true treatment effects, apply the alternative statistical methods and examine how the estimates compare with the truth. Such studies can investigate bias in the treatment effect, the coverage of confidence intervals, the correctness of P-values, and the power that different methods have to detect differences. One of us undertook and reported such a study of statistical methods for meta-analysis of rare events, and found evidence that the methods to be recommended in practice exactly are the opposite to the recommendation of Prochaska and Hilton, with the Peto method being the least biased and most powerful in situations where event rates were around 1% [3]. Notably, whilst the Mantel-Haenszel risk difference method produced relatively unbiased estimates of treatment effects, the method was shown to be seriously limited in its ability to detect real differences, as its confidence intervals are wide, giving poor statistical power. It is thus unsuitable for meta-analysis of rare events, as it reduces the chance of real increases in rare adverse events being detected, with the subsequent possibility that patients continue to be exposed to harmful effects of some medications. [1] Prochaska J, Hilton J. Risk of serious adverse cardiovascular events associated with varenicline use for tobacco cessation: systematic review and meta-analysis. BMJ. 2012;344(e2856) [2] Vandermeer B, Bialy L, Hooton N, Hartling L, Klassen TP, Johnston BC, et al. Meta-analyses of safety data: a comparison of exact versus asymptotic methods. Stat Methods Med Res 2009;18:421-32. [3] Bradburn MJ, Deeks JJ, Berlin JA, Russell Localio A. Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events. Stat Med 2007;26:53-77. Competing interests: None declared http://www.bmj.com/content/344/bmj.e2856/rr/585295
But I can explain to my 6 year old why ignoring all the trial in which there were no cardiovascular events will lead to an overestimate of risk. And applying the CV event rate for patients with a history of CV disease to calculating the NNH for patients without such a history? I don't know how to comment on that without being uncivil, and thus disrespectful of the friendship between you and our host.
If you want to be a crusader for the common good, you have to serve as a crusader for the truth, first and foremost. Distorting the truth may seem like it serves a good cause in the short run, but it always causes problems eventually.
This willl not be the first time or the last time when a drug is claimed to be absolved of a risk because the side effects(heart attacks etc) that occured in the trials, which were deemed to be good enough to get the drug on market went missing from analysis of the trials.
However, in order to definitively prove cause and effect you may wish to complete the protocol by undergoing a repeat challenge with Chantix and see if you get another heart attack. Just make sure your life insurance is paid up first. If the outcome proves causality you might advance the cause of science by leading to more restrictive labeling for Chantix.
You'll be a real Champ(ix).
As many as one-third of prospective study participants were turned away for conditions including but not limited to a history of cancer, cardiovascular disease, chronic obstructive pulmonary disease, a history of alcohol or drug abuse, major depression, panic disorder, bipolar disorder, systolic blood pressure greater than 150 or diastolic pressure greater than 95, a body mass index of less than 15 or higher than 38, weight less than 45kg, those with "clinically significant abnormalities in the screening laboratory values…” The list goes on and on.
If the trials were done like they were suppose to be done,before the drug made its debut people wouldn't need scientific proof that it dose or dose NOT cause these side effects.All I do know is . If there was a risk for me taking this drug , I would have not taken it. I don't care how miniscule of a risk it was.I did however NEED to know their was a CV risk.
Had I NOT taken Chantix mySELF, and endured the painful side effects...I'm sure I, TOO, would'v had my doubts about the VALIDITY of some of the horror stories told. But, I did...and unfortunately, can vouch, this drug is for real. The side effects DISTURBINGLY real. People don't go to the trouble and expense of getting this script w/hopes of suffering debilitating side effects. They WANT it to work!! And, it DOES touch on the smoking receptors...but, it doesn't STOP there...and THAT'S where the trouble and confusion comes in.
If Pfizer has ANY conscience, they'd PULL this drug...NOW!!!!!!
Along w/the horrendous chest pains...powerful, agonizing STOMACH spasms started...so strong they felt like contractions...then my head started throbbing...it was bizarre. I never knew a person could stay CONSCIENCE and endure such pain!! Was praying I would pass out!!! Took all my strength and several tries to get the words out to my husband to call an ambulance.
I was kept a week and underwent every test possible from head/toe. They couldn't contribute/tie this whole painful ordeal to ANYthing. They were baffled!! Put me on oxygen, morphine, antibiotic drip and catheter.
The only other times I was in the hospital was when I had my tonsils out at 5 and gave birth twice!! To be hospitalized TWICE PLUS psyche unit w/in a 4 month time frame DID rock me to the core. The point I'm trying to make is IF YOU FELT LIKE YOU WERE HAVING A HEART...I BELIEVE YOU!!! I REALLY DO!!! I FELT IT, TOO!!! THIS DRUG HAS GOT TO GO!!!!!!!!!!!!!!!!