By Guest Blogger, Redlan.
Media reporting of science is a beautiful thing to behold, and usually dreadfully and woefully wrong. Read the original study and you are pretty much guaranteed to find that the study found almost the complete reverse of what was reported.
The Daily Mail recently reported that
"Meditation 'as effective as medication' in treating depression" and the BBC followed with
"Group Therapy 'beats depression'"As effective? Beats depression?
On this one they might have redeemed themselves though, as they’re both only half wrong. What they are talking about is something called
Mindfulness-Based Cognitive Therapy or MBCT. Sounds really sciencey if we just use letters. It’s a form of CBT (
cognitive behaviour therapy) which is a therapeutic technique that aims to correct behaviours, emotions and thoughts through a goal-orientated approach. It’s actually an umbrella term for a group of cognitive and behavioural approaches, of which MBCT is one. The paper that prompted both the headlines is
Mindfulness-based cognitive therapy to prevent relapse in recurrent depression by Kuyken et al from the University of Exeter.
The 'meditation' bit comes about because the therapy uses some meditation techniques. But do not be fooled - MBCT isn’t meditation, it’s a therapy. We’ll deal with the Daily Mail headline a bit later. Never mind whose meditation techniques it uses, does it actually work? For now let’s look at the results:
The Daily Mail and the BBC don’t quite agree on how effective it is. The Daily mail says
47% of people with long-term depression who underwent the therapy suffered a relapse, compared to 60% of those taking anti-depressant drugs.
And the BBC says
The trial of 123 people found similar relapse rates in those having group therapy and those taking drugs.
So what’s going on? The Daily Mail seems to imply that MBCT is better, while the BBC says they are similar. Is 47% really similar to 60%?
The abstract says
MBCT were 47%, compared with 60% in the m-ADM group (hazard ratio = 0.63; 95% confidence interval: 0.39 to 1.04)
The Daily Mail got the numbers right. What about the stuff in brackets? The hazard ratio is similar to, but not the same as relative risk. In this study the hazard ratio is a measure of the effect of MBCT on relapse rates. The 95% confidence interval is an estimate of the hazard ratio you would get if you repeated the experiment. But there is something missing…
Why haven’t they quoted the statistical significance usually denoted as p? Did they forget? Or not think it was very important?
Statistical significance is used to test the null hypothesis – i.e. the hypothesis that there is no effect. I worked out p using
chi-squared test of significance. I got p=0.12. Normally in biomedical sciences p<0.05 is considered to be significant, and so by that standard we have a null result. In other words, for this experiment, 47% is not significantly smaller than 60%. So the BBC would appear to have got it right. Hurrah for the BBC!
The NHS website, Behind The Headlines, also
discusses this paper. It rightly points out the flaws in the BBC’s and Daily Mail’s reporting. It then goes on to give a few more details about the study. The subjects had received MBCT treatment for the previous six months and were now in either full or partial remission and taking antidepressant medication. There were then randomised to receive a further 8 week course of MBCT or treatment as usual, which meant continuing with anti-depressants. The treatment arm were also offered support to withdraw or lower medication.
So what exactly is this a study of? The rather innocent sounding (though wordy) title of Mindfulness-based cognitive therapy to prevent relapse in recurrent depression, doesn’t really do it justice.
So which one is the control group? Which event do you think might have the biggest impact amongst people with depression? Being told that you were going to get another course of MBCT? Or being told that you weren’t going to get one? Participants would have to be willing to participate in another course of MBCT before they could be enrolled in the study.
In the field of mental health, the placebo effect is very pronounced. It is typically damned hard for pharmaceutical companies to demonstrate that their latest mood enhancing wonder drug works any better than the placebo. Tell someone that they are going to receive “therapy” to make them better, and lo and behold they start to get better.
Read the literature and you will find that CBT is described as a useful addition to normal drug treatments. So bearing in mind that the placebo effect is very pronounced, what are all these treatments tested against in the control group? Sham therapy? Self help groups? Going to the pub with some friends?
Nope.
They are all compared to treatment as usual, which usually means treatment with anti-depressants. So the effect could simply be down to the “therapeutic relationship” and nothing to do with the actual therapy. It’s not the therapy that’s making you better, it’s the hour a week you have with your “specialist” telling you that his therapy will make you better, that is making you better.
Patients getting CBT do better than patients who don’t get CBT, but is it the CBT itself, or the therapeutic relationship that is making them better? It's impossible to tell, until someone devises a placebo based control group. I haven’t found one, but then I don’t have the resources to run systematic searches and analyse the results. If you have a spare afternoon, I guess you could have a some fun searching PubMed.
Now back to that Daily Mail headline…
Meditation 'as effective as medication' in treating depression
Really? Is meditation as effective as medication? Well, probably not according to the
Cochrane Library. They did conclude that meditation was better than no or minimal treatment on self-related depressive symptoms, but not as good as psychological treatments. So it would appear that patients believe that psychological treatments work better than meditation, which works better than nothing. However doctors weren’t quite so sure - the doctors only noticed a non-significant trend that meditation was better than nothing, and they couldn’t tell the difference between psychological treatments and meditation.
Newspapers and the media like to sell us simple stories. If they resonate with a strongly held social belief, then all the better. Usually the real picture is more complicated, and whether we like it or not the “experts” often do not know the answers. What is lost sometimes in the story is the fact that science is not neutral or unemotional or unbiased. It is after all perpetrated by humans. So let’s just ask a couple questions…
Why would the researchers of the paper behind the headlines, neglect to mention whether the result was significant or not in the abstract?
I wonder if they would have mentioned p if the result had been significant? It’s normal practice to state whether the result was significant or not. Try finding a paper without p being mentioned. I do find the paper confusing, because I can’t work out what hypothesis they are testing. Maybe they don’t mention p because the study appears to be designed to produce a null result anyway.
Why is CBT always tested against Treatment as Usual (TAU)? Sure it may be hard to devise and agree on what would constitute a proper placebo, but how else are you going to know whether CBT is not simply itself a placebo? I guess though you’d have to be brave to design a study knowing it could end up disproving years of training and research.
Of course, that's no reason not to do it.
------------
Redlan
----------------------------------------------------------------
BPSDB 
