Re:Maxfield & Hyer meta-analysis: A few questions

A few quick comments in response to Ricky's post.

First, you cannot make logical inferences about the quality of treatment from outcome. This involves the logical error of affirming the consequent. The logic of the argument is that (a) If treatment is done poorly, then outcome will be limited; (b) outcome was limited; therefore (c) treatment was done poorly. While it is possible that the reason for poor outcome was poor implementation, it is not the only possible reason, not the least of which is the possibility that the treatment just didn't work very well in the population under study.

Second, the results of the Devilly and Spence study are not all so bad. The Table below summarizes the within group effect sizes from the Devilly and Spence study for measures of self-reported anxiety (trait portion of the State-Trait Anxiety Inventory), self-reported depression (Beck Depression Inventory), self-reported PTSD symptom severity (the Civilian Mississippi Scale, the Impact of Event Scale, and the PTSD Symptom Scale - Self-Report), and interviewer rated PTSD symptom severity (PTSD-Interview). For anxiety, the ES in the Devilly & Spence study fell within the 90% Confidence Interval from the Van Etten and Taylor meta-analysis. For depression, the ES from Devilly and Spence fell below the 90% confidence interval from Van Etten and Taylor and similarly for self-reported PTSD symptoms. However, for interviewer rated PTSD severity, the effect size from Devilly & Spence was a whopping 2.37, which was higher than the 90% confidence interval from Van Etten and Taylor.

Self-reported anxiety
Devilly & Spence .79
Van Etten & Taylor .95 (.69-1.21)

Self-reported depression
Devilly & Spence .68
Van Etten & Taylor 1.05 (.81-1.29)

Self-reported PTSD Sx
Devilly & Spence .75, .74, .75
Van Etten & Taylor 1.21 (.99-1.49)

Interviewer rated PTSD Sx
Devilly & Spence 2.37
Van Etten & Taylor.69 (-.06-1.44)

Third, the arguments against treatment fidelity within the Devilly and Spence study ignores the fact that someone who was (a) Level II trained in EMDR and (b) not otherwise associated with the research (i.e., had no investment in the outcome) rated all available video tapes of EMDR sessions and provided a very high rating of treatment fidelity.

Fourth, the issue of treatment fidelity goes for all treatments in a study. If you compare the procedures to insure treatment fidelity for EMDR and CBT in the Lee et al. study, you'll see the two treatments were handled differentially. For example, EMDR therapists were trained by individuals who were credentialed by EMDR-I. However, with CBT, they make reference to Foa's protocol, but they did not arrange from similar training from Foa. Fidelity ratings of EMDR used a detailed checklist. For CBT, they used a single item rating the overall quality of treatment. Very different standards were utilized.

Finally, and really the reason I asked Louise about how strong her methodological position was, is that Ironson et al. altered the EMDR protocol. They added patient education from the Foa exposure protocol and they added in vivo exposure. Applying Louise's standard that changing the EMDR protocol violates the internal validity, then the Ironson et al. study has poor treatment fidelity. Yet, we don't see mention of this when the study is discussed.

Replies:

There are no replies to this message.

| Behavior OnLine Home Page | Disclaimer |

Copyright © 1996-2004 Behavior OnLine, Inc. All rights reserved.