Seems simple, yes? And so a group of Duke University researchers developed a hypothetical model to estimate the expected incremental number of adverse drug events that could be avoided once a drug is widely available. The upshot: "Requiring larger preapproval databases could be a cost-effective means of reducing adverse events in post-approval populations," they write inHealth Affairs.
As you may recall, most industry-sponsored clinical trials of new drugs are designed to determine efficacy in order to gain regulatory approval. But the researchers note that detecting adverse drug events is "rarely considered when sample-size calculations are undertaken for clinical trials." Thus, the statistical power to find side effects is "inherently a by-product" of establishing efficacy.
Their hypothetical study with 4,000 patients in each patient group improved the odds of detecting a side effect that affected slightly more than 1 percent of patients. The statistical power was 76 percent in the safety database with 2,000 patients and 96 percent in the database with 4,000 patients. They developed their scenario by reviewing literature on Vioxx and Celebrex, and suggest heart problems may have been detected sooner with larger clinical trial populations.
"Assuming that detection of a statistically significant increase in the risk of side effects would prevent regulatory approval, about 57,000 adverse events on average would be avoided in the target population with the smaller database, and approximately 72,000 would be avoided with the larger database," they write. "Thus, in the base-case analysis, an additional 15,000 adverse events would be avoided, on average, if larger databases were required to evaluate safety concerns."
In an accompanying editorial, Alan Garber, a staff physician at the VA Palo Alto HealthCare System and a Stanford University health policy professor, writes: "For most drugs, a requirement for more preapproval data will inevitably slow the approval process. Entry delays may impose only minor costs if alternative therapies of nearly equal effectiveness are available, but the consequences are much more costly if the drug in question is a unique treatment for a life-threatening condition. In retrospect, critics will claim that the delay was harmful if the drug is later found to be safe and highly effective, while the delay will be viewed as prudent policy if the drug in question is found to be harmful."
18 Comments
Yeah, us in pharma could have bigger and bigger trials (why stop at 4000? Why not 1,000,000?), but then the public would soon complain about the prices being too high. Something’s got to give people. If you want more studying of drugs before hand, be ready to pay the price to recoup those costs. This is a business after all and research isnt free. Not to mention even if we ran it in 4000 people and missed something, you’d be sure that some lawyer would be ready to bankrupt a company due to side effects. Here’s a piece of knowledge general public, all drugs have side effects and can raise risks of all sorts of potential problems. Weigh the risk of these against the benefits. Most of the people taking Vioxx probably would want it back on the market, but the lawyers saw to that didnt they.
Industry Guy,
This may sound strange to you, but why not spend money on testing medications on people who consent to partipating in trials instead of unleashing drugs with unkown side effects onto the public who never agreed to participate in the trial?
Maybe people wouldn't be inclined to sue because a loved one died had they known death was a possibllity. Stop blaming the patients by stating all drugs have risks when the public isn't even being told what that risk is. We can't weigh the risks against the benefits because we don't and you don't (if you haven't hid the data) know what the risks are.
You can't expect to keep putting crappy medications on the market and ask the public not to sue you when a loved one dies or is injured. Why shouldn't the public be compensated for participating in a drug trial they weren't aware they were participating in?
How many "helped" people are equivalent to one killed person? After all we have to balance these things out.
Another statistical unknown - Can we balance the cost of law suits from deaths and injury with the cost of additional trials? Or how many of us would gladly pay more money for our drugs if we knew the chances of it killing us or our family and friends was less? How much profit does a pharmaceutical company that hides adverse data, participates in collusion and generally is a bad actor, need or deserve? Just some things I ask myself.
Jaynesday, the number to which you are referring is the "net number to harm". It's converse is the "net number to treat" to get a benefit. This number is commonly used along with relative and absolute risk reductions.
I have one other statistical question...how much money is spent by private insurance companies, Medicare, and Medicaid to treat people who suffer from an adverse event that occurred as the result of an inadequately represented clinical trial? I can tell you this, in my mother's case Medicare payed over half a million dollars. Thank goodness my father wasn't responsible for this bill but, nonetheless, this was public monies being spent on something that shouldn't have happened in the first place. If this medical device had been adequately tried, the company would have been aware of who should and should not receive it and the tragic reaction would not have occurred.
Vioxx was fast-tracked because of the hope of a GI benefit that turned out to be minimal. There was no efficacy advantage. (Any more than ibu works better than aspirin for some folks and the opposite for others). Meanwhile it was blockbuster-marketed on an unprecedented scale, illicit promo material was distributed to docs (as described in two FDA warning letters), med sci researchers were targeted for threat and intimidation, appropriate warnings were delayed for more than a year, and tens of thousands of people needlessly died.
Industry Man blames the lawyers.
Thanks Bob Freeman, Help me understand. Is this logic valid? In order to maintain a high number needed to harm (the number of patients treated before one is harmed), it is best to have a high number treated. So to maintain a high number of treated it is best to get out there and advertise your drug. I'll shut up now.
Actually, Jaynesday, it's the reverse: the smaller the number (NNH) the higher the risk of an adverse event. If you were speaking about net number to treat, the lower the number the better.
In other words, if the NNH is 8, you see one case of the side effect in treating 8 patients than you would have seen otherwise. If the NNH is 400, you would see one case in the 400 people treated than you would have without the treatment.
Computationally, the net number to harm is 1.0 divided by attributable risk, which is the incidence rate of the side effect subtracted from the incidence rate in an untreated or control population.
In my opinion this is a red herring. If you look at the Vioxx presentations made at the FDA advisory committees the risk becomes most evident after 1 year of therapy.
Right now most NDAs are 2500 - 4000 people but only 500 for 6 months and 100 for 1 year. Since you need triple the number to detect even one case even with 4000 people and 100 for one year you would only see a single case if the rate was 3.3% or higher between 6 months and a year. If the normal background rate was even 0.3% one case would be a 10 fold increase but would always be explained away as due to chance.
Besides even if you see 15 or 20 cases of AEs they're often explained away. For example with Vioxx this is a long term cumulative toxicity and you may start with symptoms of pulmonary hypertension, (i.e. cough, fatigue, difficulty breathing, edema, heart failure) well before it's picked up on that these are simply the beginning symptoms and heart attacks and strokes are coming later. Plus you're most likely to see it in the most vulnerable population, e.g. the elderly, or in neonates whose mothers took it and they have trouble breathing (this will typically show up in the animal reproductive studies. Cardiac deaths will just be explained away as age related death. Plus these long term trials often don't have placebo controls for comparison. (It's unethical according to all those highly paid expert opinion leaders and medical societies.)
Best thing is to know the drug, predict what you're likely to see and where and when you'll see it and look very closely for it (a high index of suspicion). I'm not saying we shouldn't study more people for truly unanticipated side effects but just doubling the numbers or even increasing them without consideration of the other factors Pharma uses to hides AEs will simply give you a false reassurance. REMEMBER VIOXX'S TOXICITY WAS NOT UNEXPECTED. Merck knew what to expect based on the animal data.
Remember the Medical Ethics program at Duke is funding with drug money and was instituted after Duke got caught with major clinical study ethics violations in the late 1990's.
Justice, you are correct: the Cox-II class is a creation of marketing. The original trials were designed to show equal efficacy with NSAIDs but with a superior safety profile for gastropathies. To be blunt, their use should have been restricted to NSAID users at high risk: over 55, smokers and alcohol users.
Even that sub-population could have been co-prescribed a PPI or (heaven forbid) a generic or OTC H-2 antagonist along with a NSAID instead of an expensive Cox-II.
Right, Bob. The creation of marketing and a little cheating - like the CLASS study as published in JAMA that left out the "bad half" (for Pharmacia) of the data.
Even Bob Temple had a cow. (Interesting image.)
Industry Guy is singing a familiar tune. Blame it on the lawyers! Never mind that the companies go to great lengths to sweep adverse effects under the rug, particulalry if they are show-stoppers. Never mind that they tout efficacy and put safety in the small print. If companies behaved ethically, then they wouldn't be facing these issues, but they just can't help themselves!
First of all, these companies are fully aware of the side effects and don't disclose them all, or have false or half truthed documentation. They don't care because they can make more than they will have to pay in lawsuits. It all comes down to money. As long as they make more than they will have to pay in lawsuits, they will market it. They actually have people to estimate this! Would you expect your food, or any other product for that matter NOT be tested for safety and if it can kill you just to save a few bucks? If they spent less in advertising and more on research maybe it would be better? I can see why you are an "industry guy". WAKE UP!
Forget it, Industry Guy. My son was killed NOT because there were not enough people tested prior to "going to market" but because Lilly hid deaths and hid the lethal effects of diabetes, hyperglycemia, and death from the FDA and from doctors.
The lawyers were nothing; the lawsuit and money just blood money which I could not deal with. Justice will only come with a criminal trial.
"As long as they make more than they will have to pay in lawsuits, they will market it. They actually have people to estimate this!"
This is pretty standard in all business practice.. Just a shame that human life and loss is viewed as statistical by the pharmaceutical industry, surely human life should be valued above all else in the " human health" industry.. It is practices like the Zyprexay and Paxil scandals which makes me question the validity of its claims to care about people and their well being.. It's both ironic and disturbing...
Sorry to paste all this below, but the original article is no longer available without a fee. It will sound very familiar. My mother was one of those Oraflex killed.
It is now 25 years later. Nothing has changed. ****************************************** ARTHRITIS AT THE JUSTICE DEPARTMENT
Published: September 14, 1985
Oraflex is an arthritis drug that may have caused 26 deaths and 200 serious side effects in America. All that might have been prevented if earlier side effects in Britain had been promptly reported to the Food and Drug Administration. The Justice Department charges that Eli Lilly, the drug's maker, failed to do so. Yet last month it let a single Lilly official plead no contest, and merely to a misdemeanor. Why?
No wonder Senator Howard Metzenbaum protests that ''something has gone awry'' at Justice. The Senate Judiciary Committee plans to investigate its handling of cases against major corporations like Lilly, E.F. Hutton and General Dynamics.
In Lilly's case, how could its top officials not have realized the significance of the deaths and liver and kidney failures in Britain? These were a red alert presaging the downfall of a potentially lucrative new drug. And if the significance was recognized, how could Lilly's failure to report them, as American law requires, possibly have been unintentional? How can it merit only a single misdemeanor charge and a picayune $25,000 fine? The answers tell a sorry tale. Side effects caused by a drug can be hard to distinguish from natural illnesses. Lilly says it didn't recognize the significance of the liver and kidney damage associated with Oraflex in Britain until after April 1982, when the drug was approved for sale in America. The British drug regulatory agency didn't either, says Lilly. The British banned the drug on Aug. 5, 1982, and Lilly withdrew it from sale in America the same day.
But the Justice Department's investigators evidently didn't believe the oversight was so innocent. They noted that from early 1981 Lilly was regularly informed of adverse reactions by its British affiliate. They assert that senior Lilly officials met often with the F.D.A. before the drug's approval but ''did not tell F.D.A. of the foreign liver or kidney adverse reactions during any of these meetings.'' A Justice Department fact sheet, as Lilly itself asserts in disputing it, ''is clearly designed to create the impression that Lilly consciously disregarded medically significant information . . . and withheld information from F.D.A.''
If that's what Justice thought, then why didn't the department bring the felony charges that its fact sheet implies were warranted? Why didn't it charge all the top managers involved instead of a single individual, who, as it happens, has already returned to his native Britain? There's an awkward discrepancy between the hard words of the Justice Department's fact sheet and the soft actions it settled for.
Lilly claims the law is fuzzy as to whether foreign side effects have to be reported. So what? Wouldn't a research company want to make the best scientific use of foreign data? And if its scientists had promptly recognized the side effects of Oraflex, there was nothing fuzzy about Lilly's duty to report them.
Lilly also claims that its reporting practices on Oraflex ''compare favorably with that of any multinational pharmaceutical firm.'' If so, Americans have reason to tremble. Certain drugs - like thalidomide, which caused children to be born without limbs - pass pre-approval screening tests and don't manifest side effects until they reach the public at large. Without full and prompt reports of adverse reactions, the F.D.A. has no way of monitoring the drugs it has approved.
The prescriptions for avoiding more Oraflex scandals are obvious enough. Drug companies must diligently look for and report side effects, whether in foreigners or Americans. The F.D.A. must be tougher in insisting on getting the data it needs to protect the public. As for the Justice Department, its failure to prosecute the case its staff lawyers had built is a disservice to the drug industry, the F.D.A.'s reporting system and the public.
Jane: Bravo! The sad news is, if FDA preemption becomes law, Americans will lose their right to hold drug companies accountable when approved drugs cause harm or death.
FDA preemption will be decided this Fall in the Supreme Court case of Wyeth v Levine. Those opposed to this assault on our civil rights need to contact their U.S. representatives and senators as well as Senators Obama and McCain. Also, letters to newspapers and TV/radio might be useful.
I think/hope? there's some sensible middle ground here.
Each case needs to be evaluated individually.
Pharma is under constant pressure from shareholders to deliver profits. Whether Industry Guy likes it or not, there is an inherent conflict of interest between profits and safety.
This is not to say that attorneys don't go out of their way to feast on the fear surrounding drug safety data. Some safety issues are simply difficult to see ahead of time (even with a trial size of 4,000).
Here's an interesting thought question. In the late '90s, data began accumulating that indicated Baycol (a statin) was associated with muscle wasting (rhabdomyolysis) (see http://tinyurl.com/baycolrhabdo). The drug was pulled from the US Market in mid-2001. When would you have pulled the drug (if at all)?