The antidepressant-debunking study

Uncategorized 1 Minute

There was a news release today about a study that appears to show the uselessness of popular antidepressants.

This was reported in the Guardian, among other places. The publication can be read here.

There are problems, as summarized:

  1. PlOS is not an academic peer-reviewed journal. edit: They are in fact peer-reviewed, based on better information I have received by comments. Read the threads. They say they are peer-reviewed, but when you read their FAQ, you’ll see this: “We involve the academic community in our peer review process as much as possible. After professional staff have determined that the paper falls within the scope of the journal, and is of a minimum acceptable quality, decisions on whether to send a paper out for in-depth review are made via a collaboration between experienced, professional editors who work full time at PLoS, and academic editors who are experts in their field.”

    I’m not saying this is Wikipedia, but it’s not the same thing as a traditional journal, either.

  2. It’s one study. Beware of an equivalency between “one metastudy showed that these three or four drugs didn’t show a good outcome under these conditions” and “antidepressants don’t work.”
  3. The study measured outcomes at six weeks. That isn’t very long in a depression treatment, whether you’re using Prozac or a trampoline.

That having been said, anything that keeps family doctors from throwing the best-advertised drug at every problem is going to be helpful at this juncture. And using any kind of medication (except possibly the trampoline) without counseling is, well, crazy.

Published by substitute

nope

Published

61 thoughts on “The antidepressant-debunking study”

  3. Peer-review
    That is the definition of peer review. Every journal does that. You have a “stable” of reviewers that you can call on regularly – they are called “Academic Editors” but are not employees, just regular referees for the Journal’s papers. Those inner-circle reviewers than choose additional outer-circle reviewers (from the global scientific community) who are the experts in the field. Both the Academic Editors and the outside reviewers review the manuscripts. Then they and the authors work together with Editors – the paid journal staff – to make the paper better.
    If that is not peer-review, I don’t know what is.

    Like

    Reply

    1. Re: Peer-review
      I know how peer review works. I worked at a medical journal for years.
      Peer review doesn’t involve the academic community “as much as possible.” Peer review means sending it out to at least three known experts in the field, anonymously, and publishing or rejecting or requesting revision based on those reviews. What do they mean by “as much as possible” here? What is the role of their academic editor?

      Like

      Reply

      1. Re: Peer-review
        There is not one Academic Editor – there are hundreds and they review papers for no pay, just like any other journal.
        Managing Editor and a bunch of Assistant Editors send manuscripts to Academic Editors (choosing each carefully to match with the area of expertise). Academic Editors, who are not employees, review papers themselves AND pick additional experts to do additional reviews. Thus, each manuscript is reviewed by 3-4 experts in the field: one from the board of academic editors (who are NOT employees) and others from the broader community. The referees suggest that the paper gets published, revised or rejected. The Managing Editor makes the final decision.
        That is, in short, the desciption of peer-review. Anonymity is not a part of the definiton as there are double-blind, single-blind and open review systems, depending on the journal. All three systems are still peer-review.

        Like

      2. Re: Peer-review
        That is good to know. It makes me much more likely to accept this as a study worthy of replication and criticism.
        I do hope that it’s single-blind reviewing, though!
        Thanks for the in depth information.

        Like

      3. Re: Peer-review
        You are welcome.
        Also, just to note as there is some confusion out there, ALL of PLoS journals are reviewed in this way, including PLoS ONE.
        There is a lot of discussion about the pros and cons of different styles of review. Open review and double-blind review have their strengths. Single-blind is universally thought as horrible as it preserves the hierarchy, has a definite gender bias, Big-School bias and Anglo-American bias, i.e., compared to both Open and Double-Blind, it has much more and stonger cons than pros.

        Like

      4. Re: Peer-review
        I do understand the need for reform of the bias you listed above. As a consumer of science rather than a producer, I want the system that provides the most reduction in bias *and* a review system that allows harsh criticism with the minimum of risk to the reviewer. I suppose double-blind would be that?

        Like

      5. Re: Peer-review
        It’s not really obvious whether double-blind reviewing reduces implicit bias. You can still guess at who the author of a paper is, at least if they’re in your community, and things like writing style provide subtle cues. I attended a conference that used double-blind reviewing (which isn’t usual in my field) where the program chair basically gave the justification for it as wanting everyone’s papers to get equal reviewing time — avoiding the situation where some reviewer spends 3 hours reading their friend’s paper, and 15 minutes reading the paper by someone they haven’t heard of — rather than reducing bias.
        I think the only way to reduce bias is to get people to stop being bigoted assholes.
        Both single-blind and double-blind reduce risk to the reviewer, but not entirely, because you still get situations where some eminent personality is willing to walk into a room full of people who were on the program committee and yell “Those assholes rejected our paper!” (and he at least knows the set of 16 people that those assholes were drawn from, even if he doesn’t know who the specific assholes were who reviewed his paper — but he’s sure certain that they’re assholes, whoever they are. Not that I’ve ever seen that happen.)

        Like

      6. Re: Peer-review
        I think the only way to reduce bias is to get people to stop being bigoted assholes.
        Ouch, good luck with that.
        I agree that all possible viewpoints on the situation are horribly flawed. And I’ve worked in the sausage factory there too.

        Like

      7. Of peer review itself, one recalls that Kim’s MAGIC work was peer-reviewed, and published, by the journal Science, and, in the end, there was NO EVIDENCE to back it up.

        Like

  5. I’m not saying this is Wikipedia, but it’s not the same thing as a traditional journal, either.
    I don’t see how the section you quoted distinguishes PLoS Medicine from a “traditional journal” (by which I guess you mean a toll-access journal?).

    Like

    Reply

    1. I’m delighted by the free as in beer and speech approach of PLOS.
      My concern was that their peer review model was opaque and worded strangely, and suggested that they don’t follow the full procedure one expects at a peer-reviewed journal.
      I don’t know for sure! I haven’t worked there. But the combination of a new model in medical research publication and a contrarian, politically sensitive finding sets off alarm bells for me. I want a few more studies before I get excited about such a huge change.

      Like

      Reply

      1. My concern was that their peer review model was opaque and worded strangely, and suggested that they don’t follow the full procedure one expects at a peer-reviewed journal.
        That paragraph reads a bit like it’s intended for the public, which has no idea how peer review works and is subjected to publisher propaganda (e.g. “open access equals government censorship”ā€”yes, really). Saying that they “involve the academic community” sounds like it’s meant to reassure people who are being told that OA journals are somehow destroying peer review, or are, in fact, equivalent to Wikipedia.
        I follow the open access movement fairly closely (for kicks; computer scientists are ahead of the curve here), and if there was anything seriously wrong with PLoS Medicine’s process, I think I would have heard, even if it were only a chorus of “they suck but all the other OA journals are fine”.

        Like

    2. Quite. The bit you () quoted actually doesn’t prove that PLoS Medicine is any different from a non-free journal — proving that it is different would require more details about who PLoS’s professional staff are and how their qualifications compare to the qualifications of non-free journals’ editorial staff (within the same field).

      Like

      Reply

      1. I agree. It’s not that the open source model means anything bad at all. There are non-free journals that people in a particular field do not respect because they have poor peer review policies or other flaws.
        Considering what’s at stake, I want to be careful about results from *any* journal with both a brand new model for academic publishing and a study result that changes so much, in a way that so many people will find vindicating.

        Like

      2. What’s “brand new” about the model? With “old” journals, scientists do the real work (writing articles, and reviewing articles) for free. With “new” journals, scientists still do that work for free. The *only* difference is that the “new” journals cut out the useless, parasitic middlemen who run academic publishing companies. If there’s another difference, you haven’t demonstrated it yet.

        Like

      4. Well, I was talking about the people who *run* the publishing companies, and there are situations where even with a “professional” organization being involved (e.g., ACM-organized conferences in computer science), copy editing can be an issue (where by “an issue” I mean “something that never happens”.)

        Like

      5. Don’t get me started on the decline of copy editing or I will immediately turn into a 100-year-old man waving his cane like grampa Simpson.

        Like

      8. I’m hoping I could get them to pay me, sort of like being a professional submissive (but not really, since I’d get to be dominant by messing with other people’s writing.)

        Like

      10. And, what I meant about a brand new model is: I don’t know yet! And I’m conservative about things like medical research. Having information freely available is a great good. Not knowing about the practices of a new publishing medium, I want to know for sure that the good things about the old model haven’t been tossed out with the bad.
        A newspaper article about a study, and even the web site of the publishing organization, didn’t give me that. Hence my questions.

        Like

      12. I think we’re talking about two different things.
        The old model of “gotta pay for science information” sucked. I won’t weep for its death.
        If there is a new model, I want to know what may have changed *other than* making it free. If the answer is “nothing, it’s just not trapped under a layer of money and restricted information” then I’m delighted.
        I was concerned because I didn’t know if they’d turned medicine into a wikiLOL or a club for a particular point of view while they were doing the laudable work of making an open infrastructure. Apparently they’re not, unless my anonymous commentor is making shit up. He or she appears to have good info.

        Like

      13. Well, I’ve heard talks by the people who run PLOS, etc., and there’s no particular reason to go for any of that kind of FUD. They’re just scientists, doing the work scientists do, as well or badly as any other scientists, but unlike most, they’ve finally gotten the backbone necessary to stand up and kick out the dead weight (publishers).

        Like

      14. That’s good to know.
        For me, FUD and psychiatric research are inseparable. I treat this with pasta, which has no peer-reviewed study in its favor but a truly awesome placebo effect.

        Like

      15. Well, okay, but if the same questionable study had appeared in JAMA, you presumably wouldn’t be calling the traditional publishing model into question, right? Double standards…
        My favorite antidepressants are hookers and blow. Sometimes I overdo the blow and it wraps around to having the opposite effect, but then I soothe myself with more hookers.

        Like

      16. It wasn’t a double standard two hours ago. It was caution, solely.
        Now that people I trust have given me better information, there’s no reason for me to distrust the publisher.
        And whoa there, presumer! If a study comes out with radically different news about something in medicine, it’s one study wherever it comes from. The question of how the study was published in the first place is at another layer!

        Like

      17. And whoa there, presumer! If a study comes out with radically different news about something in medicine, it’s one study wherever it comes from. The question of how the study was published in the first place is at another layer!
        I don’t know what you mean. If the same study came out in JAMA, I assume you would still be posting about it, but I assume you wouldn’t be saying “Hey, let’s look at those freaks at JAMA and question how their review process works just because of this one paper.”

        Like

      18. I would, in fact, not change my behavior based on one study of anything. And if anyone said “hey look they just showed that antidepressants don’t work” I would say don’t change your behavior based on a news story about one study.
        Note that this is two of the three points I listed in my original post, which had nothing to do with the publishing source.
        And yes, I did question PLoS. They were new to me, and their FAQ was poorly written. That is all I knew. When I see people posting all over the internet about a shocking new study from an unfamiliar source, I don’t immediately say “wow! that must be true!” I question! And I got answers, and accepted them.
        What should have been my response to a publication from an unknown nontraditional source, other than questioning it on the Internet and getting better information?

        Like

      20. Guilty.
        I assume that every new thing I see on the Internet is horseshit until I find out differently. Particularly so when it involves health or safety, and even more particularly when it’s a topic that half the people on the Internet have strongly held beliefs about.
        BTW, I edited the original post to correct my initial reaction.

        Like

      21. Then you ought to assume that every single new thing you hear about anywhere is horseshit until you find out differently, too. In 2008, the Internet is distinct from other fora that humans use for communication how?

        Like

      22. I am really not sure what we are doing other than poking each other with sticks, here.
        The Internet is the world’s best medium for quickly spreading rumors, chain letters, and other memetic disasters. There’s a good reason why snopes.com does so well with their “inboxer.”
        One news story linked from three livejournals to one study published in a way I’ve never heard of rates about two notches above “chain letter” and slightly below “the news.” Your own filter may vary. I don’t tell you what to trust, and I doubt you trust very many things either. Do we have to have the same filters?

        Like

      23. Am I allowed to suggest more complexity in people’s analyses when they lazily dismiss things that I happen to have the experience to evaluate and have already done so?

        Like

      24. You know, the insulting language actually does upset me, no joke. I’m knee-jerk, I call people “freaks,” I’m lazy. Thanks, I feel great now.
        At this point I have corrected my original post, explained the reason for my reaction, and accepted and welcomed critical and useful information. You are indeed more than allowed to suggest a better analysis and I appreciate the one I got.
        Since I didn’t send my post to you for peer review (ha) I am reduced to explaining why I took the position I did. I’m sorry that’s unsatisfactory, but I can’t add much to what I’ve already said.

        Like

      25. Either don’t dish it out (e.g.,”turned medicine into a wikiLOL or a club for a particular point of view”) or be prepared to take it. And I really do get tired of all the Wikipedia-bashing (not from you in particular, I mean, just in general), speaking as someone who’s invested a lot of time and energy into Wikipedia and knows how well Wikipedia’s peer review process works most of the time (not that there aren’t embarrassing mistakes, but the same could be said about many scientific journals.)

        Like

      26. I don’t intend to use that kind of bruising language on you personally, and I hope you won’t do so with me again.
        It’s not something I’m prepared to take from a friend, especially over a harsh remark about a publisher, and I do consider you a friend.
        If you genuinely see me as a lazy knee-jerk bigot, then I really don’t know what to say other than “that’s incredibly depressing.”

        Like

      27. Not knowing about the practices of a new publishing medium, I want to know for sure that the good things about the old model haven’t been tossed out with the bad.
        My impression is that the biggest change in medical publishing isn’t the “real” open-access journals like PLoS’sā€”called “gold OA”ā€”but open-access repositories, called “green OA”. These haven’t done very well, because scientists don’t bother to post to repositories unless someone makes them, even if the journal explicitly allows it (and many toll-access journals do). This, despite pretty good evidence that open access, not surprisingly, increases exposure and therefore increases “impact”. Scientists are insanely conservative. The high-energy physicists may have started xxx.lanl.gov because it was the smart thing to do, but I’m sure the only reason they keep using it is habit.
        However, the NIH is about to mandate that all journal articles resulting from NIH-funded research be posted to PubMed. In the long term, funder mandates mean that the old model will work even more badly than it already does (serials crisis? what serials crisis?), because there’s no need for institutions to subscribe to journals if the articles are freely available. The journals will have to convert to “author pays” (which will really mean “funder pays”) to survive. But that really has nothing to do with peer review.

        Like

  7. Oh wow, what a bomb. Thanks for the link.
    It’s interesting because I just read a chapter of Peter Kramer’s Listening to Prozac for my Medical Ethics class. He is all about how incredibly well Prozac works, not only for depression but also for “rejection sensitivity” and whatnot, and he cites lots of cases from his own practice where, according to him, he used the drug to good effect. If I understand him correctly — this wasn’t in the particular chapter I read, but I extrapolated — he used the drug in fact to diagnose people in certain instances; I think that’s what he means by “listening” to a medicine. The book is all about how Prozac can possibly be used cosmetically, not to treat depression but to enhance personality traits. And wow, if Prozac really isn’t effective for depression (six weeks of it at least) — that really is a bomb!

    Like

    Reply

      2. I taught Kramer in a medical ethics class once. He’s just talking about his own clinical practice – i.e. it’s all anecdotal. He oversold Prozac, but his overselling was in turn oversold. He wasn’t saying that Prozac was a miracle drug, just that for some patients it had quite wonderous effects, that it seemed, in his view, to leave them “better than new.” That complicates the relationship between the doctor and the patient. The doctor sees medication as cures to disease states, and if the medication seems to make the syndrome go away, the diagnosis is supported. That doesn’t work with Prozac, because it has all kinds of subtle, global effects on character; it turns the whole of personality into a syndrome to be medicated. The rest of the book is a meditation on what that might mean for our normal ideas of character. This is now a common line of questioning, but he was one of the first to write a lot about it. It’s an interesting book, actually, though it was too long. It should have stayed a longish think-piece in a magazine.

        Like

  8. He did meta-analysis on just short-term studies because those were the ones that were submitted with the original new-drug application. He also got a bunch of other comparable studies from the literature. The idea was to do a re-analysis of the data that the FDA used to approve the drug, only using the wider pool of similar studies available now.
    This is the third time he’s done exactly the same meta-analysis. He has a big fancy metaphysical theory about the placebo effect, about its separate existence as a real, clinical thing (I personally don’t agree, but it’s not here or there), and his attacks on psych drugs are in that context. Every time he does one of these meta-studies, the profession looks at them and says something like, “even for a meta-analysis this is tendentious. You fit curves to obvious outliers, cherry-pick the data sets, and so-on. But still, it’s suggestive, given that everyone’s data is showing similarly that these drugs are not as effective as we want. It’s time for more, bigger longitudinal studies to nail this down.” Then nobody does those. Then he releases yet another identical meta-analysis of essentially the same data and it gets the same headlines, and so on and on.

    Like

    Reply

  9. Antidepressants Don’t Work
    The study only points out what many of us have been seeing. All of us have different DNA and different metabolisms. Some herbs and vitamins work better than others. The first thing that is needed is proper nutrition and a good physical exam. As the director of Novus Medical Detox, I often see patients who are on alcohol or opioids, central nervous system depressants, also taking antidepressants. When they detox they find they don’t need the antidepressants.
    This is good news because a Swedish study showed that 52% of the 2006 suicides by women on antidepressants. Since antidepressants work no better than placebos and are less effective than exercise in dealing with depression.
    There is a prescription drug epidemic and these are leaders in the list of terrible abuses.
    Steve Hayes
    http://novusdetox.com

    Like

    Reply

    1. Re: Antidepressants Don’t Work
      I agree that antidepressants are over-prescribed.
      The assertion that they “don’t work” and that “herbs and vitamins” are a better approach is entirely religious, however, and I cannot agree.

      Like

      Reply

  10. I smell CCHR (Citizen’s Committee on Human Rights) at work here, the Scientology fronth group that hates psychiatry. Granted, I do agree with them that too many meds are rx’ed to kids (Ablify for pre-teens, holy mother of god). And I do see a lot of “throw a pill at her” family doctors (like my friend’s wife who takes a prozac every three days–not the time release one for PMS either) a sub clinical level and has been doing ttis for 6 years because according to her she is menopausal, wiht no cognitive therapy or anything else–and no suggestion ot lay off the booze and pot either> Idiots.
    Anyway this journal looks like a crock, the study is a crock and while I am a firm believer in diet, nutrition and exercise to help the body do its thing, I also think that sometimes the body needs extra help in the form of neuro-supliments like SSRIs etc.
    CCHR has managed ot get a few articles placed recently and this just smells like their crap.
    Bring on the lithium crystals, Scottie! No wonder Spock was so mellow.
    Abilify for kids is BS though. Exercise, no sugar, decent parenting.

    Like

    Reply

    1. From what springheel_jack said, it looks like this guy is a legit scientist who is just very interested in the placebo effect, and not a CCHR clone.
      There are a lot of babies flying around with the bathwater; I’m just going to maintain my course and wait and see.

      Like

      Reply

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.