Re:Projective Testing

Henry,

Apparently, "What's Wrong With This Picture?," by Scott O. Lilienfeld, James M. Wood, and Howard Garb which appeared in the May 2001 Issue of Scientific American, is an extension of an earlier article by Scott O. Lilienfeld, "Projective Measures of Personality and Psychopathology: How Well Do They Work?" In this article, in the September/October issue of the Skeptical Inquirer, Lilienfeld wrote regarding Adler and early memories:

"A third early projective technique, pioneered by another protege of Freud, the Viennese psychoanalyst Alfred Adler is the elicitation of the respondent's earliest memory. Adler (1931) claimed that the first memory provides information regarding the orgins of inferiority feelings and one's means of compensating for them, although he never conducted systematic research to substantiate these assertions" (p. 33).

As an Adlerian Psychologist, I have effectively used early recollections in my work with clients for almost thirty years and am concerned when authors demonstrate such a superficial understanding of Adler and his work. As noted in a previous post, I am also concerned about these particular author's research methodology and hope serious students and colleagues take a very close look at their work before accepting it at face value. Consider excepts from an article by Gregory Meyer below.

Thanks,

Carroll

On the Science of Rorschach Research

Gregory J. Meyer
Department of Psychology
University of Alaska Anchorage

"I describe problems in an article by Wood, Nezworski, Stejskal, Garven, and West (1996b). . . . Sound criticism of Rorschach research will advance science and practice, but the Wood et al. article did not provide sufficient guidance" (Meyer, 2000, p. 46).

"Thoughtful (p. 46) critiques that bring critical issues into focused relief or appropriately warn about the dangers associated with particular methodological or statistical designs are embraced because they, ultimately, advance genuine knowledge.

"Wood et al.'s (1999b) article . . . contained many inaccurate and misleading statements. Most troubling, there is reason to believe that Wood et al. knew some of their assertions were incorrect and misleading even before they submitted the article for publication.
". . . . because of the seriousness of the issues and because available research indicates published retractions have little or no impact on decreasing the frequency with which an originally problematic article gets cited (e.g., Whitely, Rennie, & Hafner, 1994), this article details some of the salient problems found in Wood et al. (1999b)" (p. 47).

"Wood et al.'s (1999b) article contained several general points that are quite sound. Conducting research with an extreme groups design does produce effect sizes that are larger than those observed in an unselected population. Appropriate control groups are important for any study that wishes to shed light on the characteristics of a targeted experimental group and experimental validity is enhanced when researchers collect data from both groups simultaneously. Diagnostic efficiency statistics--or any summary measures of test validity--would be trusted more when they are drawn from multiple studies conducted by different investigators across numerous settings rather than from a single investigator's work. . . (p. 75).
"However, I have pointed out numerous problems with specific aspects of Wood et al.'s (1999b) article. Wood et al. gave improper citations that claimed researchers found or said things that they did not. . . .
"At worst, it seems that Wood et al. (1999b) may have intentionally made statements that they knew were incorrect. If so, these statements were then used to make plausible sounding but fallacious arguments about weaknesses in Rorschach validation research. The latter could be seen as an instances of sophist rhetoric, in (p. 76) which arguments are designed to convince readers of a conclusion, regardless of its accuracy. At minimum, whenever sophistry occurs, it stretches the boundaries of proper scientific conduct and trivializes the scientific endeavor into a caricature of the search for knowledge. Such efforts would be particularly striking if they occurred among authors who often refer to ethical principles and professional standards to make a point (Nezworski & Wood, 1995; Wood et al., 1996a, 1996b).
"At best, the authors were not sufficiently careful in their scholarship (e.g., the erroneous citations), were not aware of some key literature on a topic (e.g., the composite variables), presented a limited and slanted portrayal of relevant issues and evidence (e.g., overlooking relevant information in Cohen & Cohen, 1983; Hosmer & Lemeshow, 1989; Tilden, 1989; and Burns, 1993), and repeatedly dismissed corrective feedback (e.g., regarding their faulty z-score formula and its near-unity correlation with the correct formula). These errors and oversights are reminiscent of issues that have emerged before. For instance, Wood et al. (1996a, 1996b) criticized Comprehensive System scoring reliability and suggested that it may be poor. However, they never presented any evidence to justify that claim, and they disregarded numerous studies that negated it (see Meyer, 1997a, 1997c; Wood et al., 1997).
"Given all of this, it seems fair to conclude that even under the most benign interpretation of how Wood et al.'s (1996b) false and misleading statements found their way into print, the authors did not carefully check the accuracy and balance of their assertions and did not correct pivotal mistakes that had been identified for them. Wood et al.'s article was putatively written to offer methodological guidance to Rorschach researchers. . . . Wood et al. never pointed out a methodological strength in any of the articles they reviewed.
"The latter should be a clue to readers. Evidence indicates the same study will be seen as containing more methodological flaws when it produces results that are at odds with preexisting beliefs than when it produces results consistent with existing beliefs (e.g., Koehler, 1993; Lord, Ross, & Lepper, 1979). This effect seems most pronounced when the preexisting beliefs are strongly held (Koehler, 1993). Given that Wood et al. (1999b) ignored important corrective feedback about errors . . ., it is likely that no amount of strong evidence will be sufficient to dislodge their generally negative view of the Rorschach and its research base. Their zeal to criticize the Rorschach does not always seem to be tempered by reason or fact.
"Documenting construct validity for test scales is a slow and cumbersome process. Every individual study contains flaws or shortcomings, so it is only through the gradual accumulation of research employing different types of designs, samples, and criteria that one can confidently validate test scales. . . (p. 77).
"As the Rorschach evidence base continues to grow and develop, sound and balanced criticism of the literature will help advance scientific knowledge and applied practice. Conversely, publishing assertions that are known to be wrong or misleading can only serve political purposes that thwart the goals of science and retard genuine evolution in the field. Because of its many problems, the Wood et al. (1996) article does not provide illuminating guidance. Those who wish to have a balanced understanding of Rorschach limitations and strengths would be wise to consider other sources" (p. 78).

Primary Reference

Meyer, G. J. (2000). On the Science of Rorschach
Research. Journal of Personality Assessment, 75(1), 46-
81.

Replies:

There are no replies to this message.

| Behavior OnLine Home Page | Disclaimer |

Copyright © 1996-2004 Behavior OnLine, Inc. All rights reserved.