Eugen G Tarnow December 20 2009 01:44:16 PM
The Atkinson & Shiffrin paper is cited more than 3000 times. Yet it is incorrect. But you would not know that reading the literature. Attached is my paper that tries to set the record straight:
AtkinsonShiffrinIsWrong.pdf
Of course, the paper is probably not going to be published because it either (1) hurts people's feelings (2) is a negative publication (3) writes something everybody knows (4) goes against the dogma of the field. Or it could be because Atkinson was the Director of the National Science Foundation, President of the University of California system? You pick. Or the paper could simply be wrong or badly organized.
Here are the referee reports I got. What do you think? Feel free to post your comments. Shiffrin's comment is posted below.
Editor: We have reviewed your manuscript, and we are sorry to inform you that it does not fit our journal's main goals of advancing both theoretical and empirical aspects of psychological processes. Your paper provides a short critique of the Atkinson-Shiffrin model, but it does not incorporate more contemporary works of memory, many of which also analyzed this original model and provided new variants to understanding the complex nature of memory processes.
Editor Y: Your paper has been thoroughly read and discussed by the editors. Unfortunately, it is our opinion that the work does not reach the level of conceptual advance that would make it a good candidate for consideration as a commentary in Y Journal. Please note that this decision does not imply any criticism of the work.
Editor X: First let me thank you for submitting your manuscript to X. I was planning to invite reviews from two experts in your research area but had a unusually difficult time finding any volunteers for this paper. All I received until today is the review from Richard Shiffrin (R1), which you find below. Given that you are waiting for a while already, I decided to base this action letter on this one review in my own reading?which turned out to fit with the reviewer's impressions.
The major problem is, in a nutshell, that there is probably no audience for this contribution?at least among the readers of our journal. The issue that is addressed is too specific and technical to make for a useful historical contribution and it is way too outdated to make a relevant scientific contribution to current thinking. Moreover, the fact that no attention is devoted to all the theoretical developments since 1968 makes it impossible to judge the theoretical implications of your comment. It just comes too late one might say.
This leaves me no choice but rejecting the manuscript. I'm sorry to be the bearer of such bad news, but I hope you can understand my reasoning. In any case, thank you for choosing X as an outlet of your work, I'm looking forward to further work from your lab.
Reviewer #1, Rich Shiffrin:
As much as I like to hear that someone actually read the 1968 paper (not to mention enjoying yet another citation), this strikes me as a 'silly' paper. First, it goes on at length about a minor aspect of the details of the buffer process, many aspects of which have been fleshed out in 40+ years of research since 1968. Second it makes the point that the model is 'wrong' when that is irrelevant: All our models are extremely wrong, and are simply crude approximations that that are meant to aid understanding, and lead to further progress. Of course a 1968 model ought to be even 'cruder' than a more recent one. It is also bizarre to see that someone is hoping to publish a paper that ignores entirely the last 40+ years of research and progress in the field. Finally, my quick reading of the submission revealed a number of places where one could argue about the analyses and interpretations, but given the major problems just noted, it is hardly worth going into these.
Editor #1: I have read your paper "The Atkinson-Shiffrin model is wrong". I agree with the assertion in your cover letter that most researchers in the field know that there are problems in the Atkinson-Shiffrin model (including the authors of the model) even though it often gets cited inappropriately. The paper offers an interesting analysis of the assumptions of the model. Nonetheless, I am afraid the paper is not appropriate for Psychological Review. As you may know Journal X is the primary outlet for new theories and models across the entire field of Psychology and this paper does not offer theoretical news, even if it may have a message for the world more generally. The competition for our pages is intense, and I must focus on those papers that best fit our mission.
I'm sorry that my news couldn't be better. Indeed, you may feel that this letter is a bit unfriendly. It isn't intended as such, and I would prefer to think that my rapid response would enable you to more quickly seek an appropriate outlet for your work. I certainly wish you well in this endeavor.
Reviewer #1: The manuscript is devoted to consideration of the Atkinson-Shiffrin model (A&S) of short term memory. The author formulates that the A&S model is based on four postulates which are ill-defined and contradictory. Also, and the author declares that the four postulates are too many for a good theory. After that the author considers four "simplified models" where one of four postulates is not valid and demonstrates that each simplified model fails to describe experimental data. Therefore, all four postulates are important. However, the conclusion of the manuscript sounds: "I have shown that the A&S theory is problematic in many ways and in particular does not describe . data". This conclusion sounds like an unmotivated statement. It seems that this conclusion is just relevant to the simplified model.
Manuscript looks like a collection of contradictory statements and criticism of A&S model. It is not clear what is a goal of the manuscript, what is a novelty, what is a message to the scientific community. Also, the manuscript is poorly organised and many statements are not clear formulated. Thus the manuscript does not satisfy a standard of journal publication. I suggest to reject the manuscript.
Reviewer #2: In his manuscript, Tarnow provides a critique of Atkinson - Shiffrin theory of memory. In particular, the author cites the vagueness of definition of some concepts, and the lack of a good fit to Murdock's (1962) free recall data, as primary arguments for the critique. However, the manuscript is confused and confusing, and it falls far short of providing even incremental advancement to our existing knowledge.
The manuscript fails (rather conspicuously) to distinguish between a conceptual framework (Atkinson - Shiffrin theory) proposed to explain a particular phenomenon (free recall) and a quantitative, predictive computational model of the same phenomenon. First, as a mechanistic critique, the manuscript cites that the theory requires "four concepts to fit two parts of a curve" and that the "curves cannot be fit without all these four concepts." Tarnow asserts that "a true test of a theory is whether after fitting it to a data set, it can then predict correctly other relationships of the data." However, this criticism is irrelevant to a conceptual framework but more applicable to a predictive model. (Parenthetically, it appears from the Introduction that the author is more critical of the writing style of Atkinson and Shiffrin than the actual theory.) It is not uncommon, in conceptual theories of behavioral data, that various behavioral phenomena are described/explained using abstract concepts (such as "rehearsal buffer"). However, it should be realized that these concepts are not meant to be solid, well-defined variables in a parametric model or to correspond to actual physiological substrates. They rather serve as a useful starting point for development of more detailed, accurate theories or models. (In fact, the appeal of Atkinson - Shiffrin theory as a simple conceptual starting point, not its formal applicability, is the reason for 3000+ citations.) Therefore, the author's attempt (and his motive) to criticize Atkinson - Shiffrin theory is confused and confusing.
Second, it is obvious that all concepts, by design, are required to explain primacy and recency effects; to show that the theory cannot explain the data without one of these concepts makes no sense. In other words, the theory is designed to explain a particular phenomenon, and rendering it incomplete by leaving out an integral part of it will obviously lead to its failure. A more sensible critique would be to question if and how the fundamental concepts of a theory correspond to observable substrates. In fact, all four concepts of Atkinson - Shiffrin theory were questioned, challenged, and/or revised for the last 4 decades. Numerous other researchers have raised similar questions and critiques as Tarnow did in his manuscript (e.g. "why should we start with an empty rehearsal buffer?", or, whether Atkinson - Shiffrin model can explain other aspects of Murdock's free recall data; see, for example, various studies/models of memory by Baddeley's group and Grossberg's group, among others). Nevertheless, while providing criticisms, previous studies also provided quantitative or qualitative revisions or alternatives to the theory to explain further data that cannot be explained/predicted by the original Atkinson - Shiffrin formulation. The current manuscript, however, only points out the phenomena that the theory cannot explain in its original form (which is not novel anyway), and does not provide any alternatives. Therefore, the current manuscript does not offer any insights into short-term memory.
Reviewer #3: First, the paper is not in APA format. The figures are not placed at the end, sometimes with a strange number (Figure X), axis labels are not displayed correctly (see Fig 2) and the formatting of the figures is not according to the usual standards.
Second, the text is often unclear and sometimes not grammatical (see p. 2, "I myself is a doubter") .
Third, the logic is often incorrect. See e.g. p. 2/3 and p. 13. Why should there be no output interference effect when the last presented item is the item recalled first? After all, from the model's point of view, at the start of the recall all 4 items in the buffer are equivalent.
Fourth, the analyses and the conclusions drawn are often incorrect. For example, the comparison in Table X is clearly biased due to taking the lowest recall probability in the data (hence capitalizing on random error).
Here are some more referee reports on the next version of this paper:
Reviewer #1: This article presents challenges to the Modal Model, principally targeting the Atkinson & Shiffrin (1968) paper.
The main weakness of this article is that it is unbalanced and disconnected from the relevant existing literature. For this reason, its value is unclear in that it is redundant with some pre-existing criticism and it challenges an already weak model. Models that succeed (e.g., by bypassing the entire notion of dual memory stores and buffers, or by specifying what these constructs mean) are not mentioned, which is puzzling. The strengths of the model are not mentioned either, which is equally puzzling.
Page 3, first paragraph: I do not see the contradiction. Can't one control something one can't observe?
Page 4, strange criticism of random selection of items to drop out of the buffer. Why should the randomness originate from random fluctuations in firing rate? Wouldn't this kind of (approximately) random process be more related to fluctuations in stimulus properties, context, etc.?
Page 4, "Biochemical evidence..." - how is this relevant? LTP has nothing to do with the psychologist's notion of "long-term memory", including the Modal Model. This is simply a terminological mismatch. Again it would help to pay attention to what other people have written. Interestingly, LTP does not seem to correspond to any relevant time scale in observed behavior, so it might be a red herring anyway (as many people have suggested).
Large number of parameters needed to fit the SPC - Great, so see the many ratio-rule models including SIMPLE.
Page 5, "working memory" is mentioned but never defined. This is a very controversial construct, so it should be treated with care and precision.
The modeling exercise is interesting and might be instructive, but it is unclear to me whether or not this is novel. I'm sure many people have done this before and very likely published these kinds of simulated exercises.
Page 12, "intermediate item" - use the conventional term, "asymptote". Which item are you using?
The model is not fully specified.
It is unclear why Cowan (2000) of all papers is used to set the buffer size.
Reviewer #2:
Specific
Page 2, para 1, line 4: no need to point out that Izawa was a former graduate student in same institution as Atkinson & Shiffrin.
Page 2, para 1, line 6: use of the word "apparently" raises doubt as to the reliability of the statement.
Page 2, para 3, lines 1-2: Informal style for this type of publication.
Page 3, para 1, line 1: "this theory has to be well defined" replace "has" with "have".
Page 3, para 1, line 1: suggest replacing "problem" with "issue".
Page 3, para 2 and 3 - (for PB) check that his interpretations of the A&S definitions are justified.
Page 3, para 4, attempting to advise the reader on the attributes of a theory could be considered patronising, suggest rephrasing.
Page 3, para 4, line 7: "She must have misunderstood...used definition " seems sarcastic and inappropriate for this publication.
Page 3, para 5: this para suggests that we must always have a partially full RB. However, this need not be the case, especially in the case of a verbal memory task, in which participants are given time before beginning the expt without any verbal noise.
Page 4, para 3, line7: "introduces" should be "introduce".
Page 4, para 3, line 11: "is" should be "are".
Page 4, para 3, lines 10-11: Operational definitions clearly are helpful in defining theories and the literature indicates that many publications use STM and others STS, whereby STM is the process and STS is perhaps the venue. It's not clear why the author objects to the A & S model using this STS and STM separation.
Page 4, para 4, line 2: "Bending concepts into a pretzel as is done by A & S", this seems a very sarcastic manner of communicating the authors views to readers and is not helpful.
Page 4, para 4, line4: "predict" should be "predicts".
Page 5, para 2, line 4: "ingredient" should be "ingredients".
Page 5, para 3, line 3: if the presentation rates are 0.5 and 1 second, but the number of seconds allowed to pass between list items is 2 (line 1 of same para), then there are 1.5 and 1 seconds not accounted for in your explanation of the Murdock methodology. If these missing 1.5 and 1 seconds are can be accommodated by the ISI, this should be made clear
Page 7 , para 1 , line 1: some text missing before "LTS".
Page7, para 1, line 4: it's unclear how a first in first out algorithm would generate probability recall rates of 50% for items 1-6 and 100% for words 7-10. This needs more specification.
Page 7 , para 1, line 5: it's not clear why starting with a non-empty buffer would generate the model data in figure 1D.
Pages 8-10: graphs need legend specification, beyond the first graph.
Page 12 , para 2, lines 1-3. This text refers to table 1. Where are the starting numbers to apply your definition of primacy strength? It's not clear how the values in the table are generated.
Page 14, para 1, lines 1-9: the example is specific to list positions 7 & 8, effect are middle serial positions is likely to be quite different.
Page 18, para 1, lines 1-3, more explanation needed to accompany table 3.
Page 23, para 1, lines 1-2: use of the phrase "it is a mystery" could be considered inappropriate.
Page 23, para 2, line 5: "The latter include the failure" - the latter what?
General: This paper focuses on trying to disprove the Atkinson and Shiffrin model, but does little to consider other theoretical models that have been published since the seminal Atkinson and Shiffrin paper. There is little or no mention of the Working Memory model (Baddeley & Hitch, 1974), the Act-R model (Anderson & Lebiere, 1998), the SIMPLE model (Brown, Neath & Chater, 2007), the primacy model (Page & Norris, 1999), Positional Distinctiveness model (Naire et al, 1997) etc. The current literature on FR is not fully explained by the A & S model, and consequently more recent models have been suggested, adapted and cast aside in light of further research. It is not clear to me, why the author is attempting to disprove a model which many other researchers feel has been outdated many years ago.
In addition the writing style of this paper appears somewhat sarcastic through out. The author of this paper typically uses the first person in this paper, when it is common practice to write articles in the third person. This article is also written in an informal and perhaps more conversational style inappropriate for journal articles.
Fifth, the phrasing and the tone of the arguments are inappropriate for a journal such as X
All in all, the problems are such that I have to reject your paper for publication in X.
Editor #2: As you might have seen,X is a fine place to publish controversial articles. But, as you wrote, it is true -- we all know that the Atkinson-Shiffrin model is wrong. And we all know that math models aren't theories. The A&S model was a step in a progression. And math models are one of many tools in our toolbox. So, I will not send this article out for review. But thank you for considering X for the publication of your research.
