How is this possible? A review of unpublished studies submitted to the FDA by drugmakers found many of the meds have little or no effect on patients, according to a new review of the previously unknown results, which was published in this week's
New England Journal of Medicine (subscription may be required). The upshot is that docs and patients are getting a distorted view of the effectivness of such drugs as Effexor, Zocor, Cymbalta and Paxil.
The researchers looked at reviews from the FDA for trials between 1987 and 2004 on 12 widely used antidepressants involving 12,564 patients, and whether the research was published. Those studies that were published were then compared against FDA versions. The found that the results determined whether and how studies were published. Most of the studies that were not positive were not published or they were published with a positive spin, such as emphasizing positive secondary outcomes when a primary outcome proved negative. While drugmakers don't have to publish all their studies, the researchers identified unpublished studies by comparing databases of medical journals with documents filed with the FDA.
In discussing their findings, the researchers wrote that the FDA deemed 38 of 74 studies to be positive and all but one of those studies was published, the researchers said. The remaining 36 were either had negative or questionable results; of those, 22 weren't published. Of the 14 that were published, 11 were mischaracterized as positive, according to the researchers, who were led by Erick Turner, a former FDA reviewer who is now an assistant professor of psychiatry, physiology and pharmacology at Oregon Health & Science University and Medical Director of the Portland Veterans Affairs Medical Center’s Mood Disorders Program.
"We found a bias toward the publication of positive results. Not only were positive results more likely to be published, but studies that were not positive, in our opinion, were often published in a way that conveyed a positive outcome. we found that the efficacy of this drug class is less than would be gleaned from an examination of the published literature alone," they concluded.
"We cannot determine whether the bias observed resulted from a failure to submit manuscripts on the part of authors and sponsors, from decisions by journal editors and reviewers not to publish, or both. Selective reporting of clinical trial results may have adverse consequences for researchers, study participants, health care professionals, and patients."
107 Comments
Ed,
Are you serious!!! This has been known since 2004. The Congressional Hearings revealed this, This info isnt new,.. just refurbished...
What is the percent of those no longer with us in that time frame, that's what I would like to know........
The quote is "Little or no effect on patients" that means it couldn't have harmed anyone either... Right??????
Brian,
You need to get a life..
Quote from Kettle: What say you Pot?... You'll have to give all that money back!!! Who to blame now?...
brian,
Got me baffled!!!! English Please....
What a suprise: Negative results aren't published. Wow. I better go write some publications about all those experiments I did that failed. Maybe a journal will accept them now that this "groundbreaking" story came out.
Nathan,
If you are truly a scientist, maybe you should reconsider comments, Sarcasm doesnt do you any justice...
You know who
If you agree the drug has no effect then how can you blame it for someone's attempted suicide. You didn't tell the whole story here..I'm convinced there is a genetic component to mental illness and then there's impact of parenting....
By the way Ed -- watch your titles. "Clinical Trials Exaggerate Antidepressant Benefits" isn't an accurate statement what-so-ever based on the story your wrote. A better statement is: "Publication of Clinical Trial Data Exaggerates Antidepressant Benefits".
Nathan, I think Ed Know's he's doing......
By the way Nathan,
I noticed you had no Rebuttal to my comment.. Can one assume that I am correct that you have absolutely no conception of the serious adverse effects children endure while on SSRI's? Or would you still rather accuse me of being a Stalker?...
Why were they on antidepressants? Doesn’t sound like the first thing you would try. I would expect their behavior was in a long term decline. If that decline continued and they were treatment resistant you might expect a bad outcome while on a SSRI. I don’t expect everyone would understand this..
Brian
Have any Scientific data to back that up!.. Or would you rather be one of those individuals who would prefer to state that its a chemiacal Imbalance..
Donna, I made a very simple observation. Do you think this title makes sense? Someone reading this title would think that pharmaceutical companies are exagerating the results of an individual clinical trial. They aren't. The scientists at the company simple choose to write articles about those trials with positive outcomes.
The more I think about this story, the more it irks me. This is a non-story. Drugs fail for many, many reasons. Maybe the drugs were tries at too low of a dose. Maybe they were tried at too high of a dose and side too many side effects were observed. Maybe the endpoint was poorly choosen.
Lisa, your thoughts aren't worth the imaginary paper that you are typing them on. Get a life and take your comments elsewhere. ----- Lisa wrote: By the way Nathan,
I noticed you had no Rebuttal to my comment.. Can one assume that I am correct that you have absolutely no conception of the serious adverse effects children endure while on SSRI’s? Or would you still rather accuse me of being a Stalker?… v
For those of you that haven't written up a scientific article for publication, I'll make you aware of a few things: 1) Journals don't accept an article just because it is written. Journal editors scrutinize it and submit it to reviewers who further scrutinize it -- very frequently rejecting articles that aren't conclusive or are poorly written. 2) Scientists write scientific articles -- not public relations representatives. 3) Scientists generally don't waste weeks (or months) of their time writing up articles that don't prove anything and probably won't be accepted by the journal. A negative result is generally meaningless in science. Do you think Thomas Edison wrote up articles about the 900 filaments that failed to light a light bulb?
Now I'll ask again: Is it any surprise that the negative clinical trials were not written up as publications?
Here's one example: The Paxil trial may have tried doses of 10, 25, and 50 mg. The 10 and 25 mg dose failed to show an effect. The 50 mg dose did show an effect. Why should anyone waste their time writing up a journal article explaining why the 10 and 25 mg dose failed? It's obvious that drug levels just weren't high enough to observe an effect.
OK guys enough of the petty squabbling...
Some people might like to downplay and dismiss this article (ie. Nathan and Brain) , but it actually is quite an important one.
And the fact of the matter is, many campaigners , such as myself and others have known this for years, but that doesn't mean it was "public knowledge"... It has got a lot of coverage in the mainstream news , deu to this research, and that can only be a good thing..
On the subject of publishing trials with positive outcomes only and not publishing trials with negative outcomes, anyone who argues that this behaviour is good for patients and the public at large needs a dose of reality...
It is deply harmful and deeply shameful, and there should be a law against it...
> Someone reading this title would think that pharmaceutical companies are exagerating the results of an individual clinical trial.
> emphasizing positive secondary outcomes when a primary outcome proved negative.
Nathan, If you neglect the (negative or neutral) primary endpoints, and focus only on postive secondary endpoints (as mentioned above), then this can qualify as inappropriately exaggerating the overall results of the study.
All you can say from the experience with SSRI clinical trials is that it is difficult to prove benefits / uncover risks in clinical trials of the size currently attempted. As we know, the true impact of a medicine becomes apparent in the larger population. If you want to get to this answer faster, it sounds like you need bigger trials. The problem is; who pays for these trials and who will volunteer?
Chris & Truthman: I understand how this APPEARS to be a smoking gun. But I'll state again why I believe it is not: Let's say I do a clinical trial with Paxil at 10 mg (once per day), 20 mg (once per day), 20 mg (twice per day), and 50 mg (twice per day). That's a total of 4 trials. Only the 50 mg dose works. Are the other three trials counted as "negative" evidence that the drug doesn't work? Of course not.
Should I write up journal articles about the 3 trials that failed? It would be a waste of my time. Maybe you can find the spare time to write them up.
One more note to Truthman: Publishers are independent of pharmaceutical companies. Even if I tried to write a paper about a failed experiment or an ineffective dose of a compound, the journal would likely reject it. Or (best case scenario) it would be published in a low-quality journal -- maybe the Scandinavian Journal of Psychology would accept it...
Nathan,
"Scientists write Scientific Articles-- Not Public Relations Represenatives"
If you truly believe this, then I have The Brooklyn Bridge Id like to sell you!
Interesting Read: http://www.healyprozac.com/ghostlydata/default.htm
The underlying biology around depression still in the process of being understood. One can't predict which patients will respond to SSRI's modification of serotonin. It is true that there is a large segment of the depressed population that will not respond to this therapy but who they are is hard to predict. In practice these patients may be seen to get worse and the question then is do we take them off medication entirely? Research into this area needs to continue but most companies don’t see any incentive given their experience with previous clinical trials. There are other mechanisms related to, glutamate substance P and CRF and other hormones but it has been tough to bring any of these therapies forward..
Another Interesting Read: Critical Docs: http://www.healyprozac.com/trials/criticaldocs/default.htm
For those of you that haven't read the article, here's a few quotes from the article in question that are pertinant to this discussion:
"We wish to clarify that nonsignificance in a single trial does not necessarily indicate lack of efficacy. Each drug, when subjected to meta-analysis, was shown to be superior to placebo. On the other hand, the true magnitude of each drug's superiority to placebo was less than a diligent literature review would indicate."
"Because we excluded articles covering multiple studies, we probably counted some studies as unpublished that were — technically — published. The practice of bundling negative and positive studies in a single article has been found to be associated with duplicate or multiple publication, which may also influence the apparent risk–benefit ratio."
"It might be argued that some trials did not merit publication because of methodologic flaws, including problems beyond the control of the investigator. However, since the protocols were written according to international guidelines for efficacy studies and were carried out by companies with ample financial and human resources, to be fair to the people who put themselves at risk to participate, a cogent public reason should be given for failure to publish."
There is some truth to this last statement. But all clinical trial data, published or not, is publicly available, right?
Realistically, shall we now consider a new perspective besides those from government agencies, research community, and pharmaceuticals in this matter? The debate can go on and on, but many people are awaiting for results if there is any. The current reality may dictate that experiences from patients and physicians count in determining the exact side effects and effectiveness.
That was how HealthLat came into being. Patients and physicians are welcome to report their experiences and check how a treatment has been used by others in http://www.healthlat.com
The above post leads you to a page with this statement: "Drugs that work for a few in clinical trials may not work for millions in the market. Side effects and interactions may not be known for a long time. Only individuals experienced it know it first."
While this is a distortion of the fact that MOST folks do well on approved drugs and a small percentage (still a large number) do not
The solution is bigger clinical trials with more patients
who pays and who volunteers??
Nathan has raised an important issue that has been known for years: clinical journals have a negative bias against studies that don't show clinical improvement. Editors want "breakthrough" studies that call attention to their journal, not studies that are equivocal or show little if any clinical benefit. And, the dirty little secret is that editors have their favorites who get positive peer reviews based on reputations and affiliations, and if an editor doesn't know you, you don't get an expedited review. An editor also knows which ones of his/her reviewers will give favorable reviews (in general) and those who won't. An editor can therefore kill or facilitate an article that he/he wants published.
Years ago, a journalist was allowed to sit in meetings of the editorial staff of the NEJM. (This goes back a long time and I don't remember where it was published: Atlantic Monthy or New Yorker, perhaps). Publication reviews just aren't that objective.
Finally, some researchers took articles that had been published in referred psychology journals, changed the name of the authors and their affiliations to lower-tier state universities and sent them out for review. Guess what: the majority of studies were rejected. (I forgot where this was published but it is from the distant past.)
Brian, The link below hurts every arguement you have attempted to make http://clinpsyc.blogspot.com/2008/01/antidepressants-hiding-and-spinning.html
Jane, No, This web site deals with the controversy of how the results are viewed interpreted and reported to the lay public. The data have always been there for those in the field.. The question is the interpretation given that some folks and perhaps a majority receive a benefit from SSRI's. The question remains, what is the underlying causes for all types of depression?
BTW: Why do you not view the information on your web site with the same suspicion Science is always a battle between factions..It is not designed to end but you have to understand its principles first
Nathan wrote: 1) Journals don’t accept an article just because it is written. Journal editors scrutinize it and submit it to reviewers who further scrutinize it — very frequently rejecting articles that aren’t conclusive or are poorly written.
Nathan--take a look at a recent article by Roy Poses on HealthCare Renewal blogsite: http://hcrenewal.blogspot.com/2008/01/antipsychotic-drugs-for-depression.html#links
If you read the article carefully, I think it trashes much of your "integrity" argument.
Nathan further states:--One more note to Truthman: Publishers are independent of pharmaceutical companies. Even if I tried to write a paper about a failed experiment or an ineffective dose of a compound, the journal would likely reject it. Or (best case scenario) it would be published in a low-quality journal — maybe the Scandinavian Journal of Psychology would accept it…
Read the article I referenced. Again, the integrity, and thus the validity, is questionable when the EDITOR is also a co-author! Perhaps you are remembering a time when science, scientists and researchers were not for sale. While there are some who also remember such days, and practice/write accordingly--they are NOT the majority.
Brian,
I don’t see much difference we have. As you pointed out, without funding and volunteers, large clinical trials are impossible. So the public, lay or not, have no choice but to rely on the (sometimes conflict-of-interest prone) small clinical trials. Yes, the experience information is in the field, as it has been since Vioxx was there for 5 years and taken by 20MM people.
Now what HealthLat does is to collect the information and make it useful as complementary information in decision makings. For the many good drugs, people get a peace of mind; for the potential bad, people are alerted. Another thing, HealthLat reports consider not only the self-interpretation of a drug but also any new conditions or symptoms after.
You might call vioxx a dangerous drug but more people have been injured by aspirin. Individual reports of drug issues are not reliable and are also biased by the potential for tort litigation. Money drives this process and drives all processes. This is a complex area and many have continued tightening the grip on pharma but to what end? Scrutiny of publications seems like a red herring. Let's deal with the actual problem that pharma can't operate under your strict rules.
Melody, I'm a scientist at a big-pharma company. I guess you'll just have to take my word on it. Editors of journals are not "paid off" by the company. Scientists write the articles (not public relations people). I've written several myself. Here is the process we go through when publishing in big-pharma: 1) Do the science 2) Patent the science (when possible) 3) Write the paper 4) Submit it to legal clearance (company lawyers) 5) (for clinical studies) Submit to internal review board/clinicians/public relations 6) Submit it to the journal 7) The journal sends it to the reviewers (experts in the field) 8) The reviewers recommend changes or acceptance or rejection.
As Bob pointed out, editors like to accept articles that make their journal look good. NEJM is probably not going to accept a bunch of articles about clinical trials that didn't work. That doesn’t mean the editors are in some sort of under-the-table deal with the company. It simply means that they are looking out for the best interest of the JOURNAL, not the public.
I think we're all missing the point. Of course scientists like to publish things that work, not things that fail. The point is that ALL clinical trial data (the good, the bad, and the ugly) is available to the public, the FDA, and anyone else interested. Whether or not the data makes it into a journal is irrelevant.
Nathan, you said it far better than I. Great post
Melody, In specific response to the link you mentioned: The link says that there was a article with Charles B. Nemeroff as a lead author in 2006 while he was also editor-in-chief of Neuropsychopharmacology. I was very surprised at this. So I decided to check it out for myself. Sure enough, it's correct. I don't know if this is a common practice or not. Certainly it's a conflict of interest and it seems rather inappropriate to me.
Non-the-less, in general, the publication process favors articles that describe SUCCESSFUL experiments/studies. The failures (unless they are catastrophic failures) just don't generate much interest to the scientists, the journal editors, or the readers.
Natan, Brian and other spinners for Big Pharma
I'm sorry but you guys are wrong on several counts. 1) ghost-written, PR based research gets published all the time. 2) these unpublished anti-depressant studies were not available to the field until recently and a lot of patients and docs were lied to in the 1990s as a result. 3.) the sad fact is your employers wildly overstated effects of anti-depressants as a result and still do, check the recent dinging of wyeth by the fda. 4.) how do you look at yourself in the mirror in the morning? 5.) maybe you should start reading my site in addition to ed's.
Nathan...
I think you are missing the point i was making. Of course i understand why negative trials are not published. Of course it would be of no benefit to a pharma company to have them published,(at least commercially) so why would they?
But i disagree about your opinion of independent publishers( and I presume you were refering to medical journals such as the Lancet , BMJ , New England-Journal etc?)
Independent publishers don't publish negative clinical trial date for three main reasons.
1. Some journals are heavily influenced or affiliated with the Pharma Industry (bias opinion is rampant as is conflict of interest).
2. Most negative trials and data on Drugs are concealed by drug companies and they do not have to disclose this data (for commercial reasons/trade secrets etc)
3. And because of the reasons above, even if independent researchers wanted to clinically trial a drug, it is difficult to get funding , so independent(non-biased) trials are rare (Thus few ever make it to publication).
Let me put it this way...
Should it take a Vioxx, a Paxil , an Avandia or a Paxil catastrophe before the real risks are made public?...
Nathan,
Please show us where ALL clinical trial data is available to the public.
My understanding is that all protocols are now required to be registered (only since a few years ago) but that there is no requirement to make results publicly accessible.
Bear in mind as well that trials are also conducted outside the legal jurisdiction of the US.
Oh and one last thing...
I would hardly call a "clinical trial of an SSRI a worthy endeavour...(ie. attempting to treat psycho-emotional-spiritual manifestations with drugs is not only useless , it is severly damaging to patients, particularly if continued long term).. The human condition does not compose of neurons, synapses and chemicals..And the brain is not a computer hardrive which needs to be interfered with.. We are talking about people here... (people with identities, emotions, feelings and thoughts..not robots..)
And on the subject of science... Pychiatric drugs are completely unscientific, as there is no way of measuring "chemical"(serotonin) levels in the brain or the body. And also results can be interpreted and doctored favourably very easily in these trials; due to the nature of the "condition" they attempt to treat, and also the structure of the trials themselves( washout, recoding of side effects etc) ..
SSRI clinical trials are about as scientific as banging a faulty TV set with your fist to make it work... Their premise and purpose is wholly unjustified, and the only reason SSRI's exist is because they make a lot of money for the pharmas.. They are no more effective than the Tricylics , they were launched in order to tap new markets (PTSD, OCD ect). And they were marketed as better because the pharmas needed new drugs to patent ... They have done nothing to further the cause or understanding of "mental illness", as a matter of fact they have put this understanding back 50 years...
Brian , you asked the question..
The question remains, what is the underlying causes for all types of depression?
.. I suggest you read some of Dorothy Rowe's literature on the subject.. maybe it will give you an insight...?
or better still, maybe it will give you the answer...
http://www.dorothyrowe.com.au/
(If Doctors and psychiatwits prescribed her books to patients and not their pharma-death-pills then possibly the treatment of depression would be revolutionised)
Truthman, yes, the evidence is that tricyclics and SSRIs are essentially equally effective. The side effect profiles are different and in favor of the SSRIs, all things considered.
I've noticed over the years that many drug classes are promoted on the basis of patient preference rather than clinical benefit (which, due to the nature of many clinical trials is done for the purpose of establishing equal efficacy)
Philip, I'm glad you consider me to be a "spinner" for big-pharma! I only wish my employer recognized me as such. I'm a non-managerial benchtop scientist. As as benchtop scientist, I have the opportunity to view the science being done firsthand -- and what is being written about it. I can assure you that most of you here don't know what the hell you are talking about. If you want to know how things work in the pharma industry, then try getting a job here. We are always hiring.
Just don't expect great payoffs. That's reserved for the upper management. Most of us get virtually nothing beyond my monthly paycheck and a yearly miniscul bonus.
Robyn, You make a good point -- lots of (early phase) clinical trials are now being done in India and China. What happens to that data? I don't know.
I can't vouch that all data IS being made publicly available. I'm saying that it SHOULD be publicly available. But that's very different than saying that all data should be put into a peer-reviewed journal. That's just not going to ever happen for the reasons I've outlined in the previous posts.
Truthman (sic)
psycho-emotional-spiritual manifestations are all caused by neuron and neuro transmitters interaction. I admit it's complex but with you it's probably less complex..
Philip Dawdy
I have made a difference with people w/ mental illness. You are focused on the treatment resistant population of which there are many. I am trying to help them too. It took years of work and we were very cautious before attempting to treat humans. To put us on an equal frame. I would like to have your free speech regulated as it could harm mentally ill people, that you control, who need treatment but listen to you instead.
BTW to all..writing papers is a huge amount of work….the glory is fleeting..I wouldn’t publish any of the 10,000 things that I now know don’t work..
"Bob Freeman Truthman, yes, the evidence is that tricyclics and SSRIs are essentially equally effective. The side effect profiles are different and in favor of the SSRIs, all things considered"
... just to correct you, "the evidence is that tricyclics and SSRIs are essentially equally INeffective"
For a vast majority of people who take SSRI's, they cause a littanty of side effects which are just as undesirable and intolerable as the Tricylics... This "class" of drugs can be worse in a lot of cases for a lot of people... And if as you say the SSRI's are equal to the tricylics in efficacy then why were they promoted as much safer and more effective option? ( could it be for drug patents and pharma profit reasons perhaps?)
"Brian Truthman (sic)
psycho-emotional-spiritual manifestations are all caused by neuron and neuro transmitters interaction. I admit it’s complex but with you it’s probably less complex.."
Just to correct you again Brain... To say that psycho-emotional-spiritual manifestations are all caused by neuro transsmitter interaction is the height of ignorance...(and totally arrogant) It is text-book pseudo-science psycho-babble nonsense...
Many things can cause anxiety, depression OCD etc... Trauma, inability to cope with life, stress, grief, dissapointment , fear etc .. These can all lead to didorders and dysfunction of the mind and personality...
Are you a psychiatrist Brian?... Do you work in the drug marketing industry... If so, then i'm not at all suprised by your outlandish opinions ... But i am dismayed at your narrow vision...
Truthman (sic)
What do you think (sic) goes on in that mass of jello in that watermelon above your shoulders. Huh??..what do think it's smoke and mirrors up there??? Every thought , emotion and stupid idea is linked (eventually) to a mass of synapse firing and neurotransmitter release. I'm not saying anyone can pin down the exact cause and effect but somewhere My very words are causing a neuron to fire in your head. Trauma, inability to cope with life, stress, grief, dissapointment , fear etc .. all lead to disorders and dysfunction of the mind and personality which can be eventually linked to a mass of neurons firing and disfiring. BTW Stay away from Tom Cruise..he's no good for you
[...] Pharmalot: Clinical Trials Exaggerate Antidepressant Benefits [...]
Brian Truthman (sic)
What do you think (sic) goes on in that mass of jello in that watermelon above your shoulders. Huh??..what do think it’s smoke and mirrors up there??? Every thought , emotion and stupid idea is linked (eventually) to a mass of synapse firing and neurotransmitter release. I’m not saying anyone can pin down the exact cause and effect but somewhere My very words are causing a neuron to fire in your head. Trauma, inability to cope with life, stress, grief, dissapointment , fear etc .. all lead to disorders and dysfunction of the mind and personality which can be eventually linked to a mass of neurons firing and disfiring. BTW Stay away from Tom Cruise..he’s no good for you
Yes, Of course I am not disputing this.. But in a way you have made my point for me...
"Trauma, inability to cope with life, stress, grief, dissapointment , fear etc .. all lead to disorders and dysfunction of the mind and personality which can be eventually linked to a mass of neurons firing and disfiring"...
So there you go.. The human experience (fear, trauma, emotion etc)_happens BEFORE the synaptic (brains) reaction... So to try to treat someone who's problem is primarily on an emotional/human experienc level with a drug like an SSRi is treating the symptom of the problem and not the cause! .. Which is not only unscientific and ineffective it is downright dangerous!..
For Example, if someone happens to develop clinical depression in later life from the suppression of feelings of being abused or neglected as a child, the chemical changes (such as a possible drop in dopamine/serotonin production) which take place in the brain are as a result of the individuals mood( and emotions)infulencing this ..
So how the hell is prescribing an SSRI useful in a scenario such as this? ..
Prescribing an SSRI for a Symptom , is a chemical band aid, it does not address the root cause of the problem, which, with depression and Anxiety is usually related to the psychology of an individual...
Pages