As self-aware beings, we humans seem to be in a constant dance between our information storage faculties and our own assessment of those faculties—I think I remember that familiar-looking person’s name, but how sure am I?  This dance guides our behavior.  Sometimes we act on what we remember (“go say hi!”).  Sometimes we don’t (“I’m just not sure enough to say hi”).  And often we seek further information from the environment, for example by consulting a friend or a smartphone.

In the past 20 years, cognitive psychologists have come to appreciate the extent to which performance on a memory test is a joint product of both memory and what is called metamemory; namely, our own feelings about our memories.  That is, a person’s overt response to a memory question reflects not just the process of memory retrieval, but also judgments about how accurate the product of that retrieval is likely to be (confidence). Whenever we query our memories, we make a decision about whether to volunteer a recollection (“oh, hello Fred!”) or withhold it (“darn, was it Fred or was it Siegfried?”).

Cognitive psychologists have made good progress in figuring out what drives people’s’ confidence in their memory, and this progress is advanced further in a new paper by Pansky and Goldsmith in the Psychonomic Bulletin & Review.

Prior research has shown that people tend to use what is called retrieval fluency (i.e., the ease with which an answer comes to mind) in estimating the accuracy of their responses.  The two experiments by Pansky and Goldsmith furthermore show that people’s estimates seem to be influenced by the retrieval fluency for a particular question relative to questions they answered earlier in the test.

Let’s just focus on their Experiment 1.

here is the basic procedure: participants took a multiple-choice test of 30 general knowledge questions (e.g., about geography or history).  On each question, participants picked one of the four alternative answers, then estimated the likelihood that their choice was correct (we’ll call this their confidence rating), and finally decided whether or not to volunteer their answer to be counted for points toward a small cash reward.  They would only get a reward if they volunteered more correct responses than incorrect responses, so this motivated people to volunteer only those responses they were highly confident about.Here’s the key experimental manipulation: for half of the participants, the first 10 questions were really hard (hard-start condition); for the other half of participants, the first 10 questions were really easy ones (easy-start condition).  For both groups, questions 11-30 were all medium difficulty, and those are the ones we’re interested in.

Here’s the key result: people gave higher confidence ratings to the medium questions in the hard-start condition (.64), as compared to the easy-start condition (.52).  This occurred despite the fact that there was no significant difference in actual memory performance on the medium questions for the hard-start (.55) versus easy-start condition (.52).  That is, the difficulty context in which people encountered the questions affected their metamemory independent of their memory.

This result is explained in terms of relative retrieval fluency.  If you just finished wracking your brain to answer 10 really hard questions, and now you suddenly get a long string of questions of medium difficulty, it feels like the answers come to mind much more fluently by contrast, and thus you’re more confident in your answers.  Conversely, if you just finished breezing through 10 really easy questions, and now you suddenly get a long string of medium questions, it feels like the answers come to mind much less fluently by contrast, and thus you’re less confident in your answers.

The difficulty of the initial questions in the test seems to set our expectation for what retrieval should feel like, and that expectation influences our confidence in our answers when we encounter easier or harder questions later in the test.

So what about participants’ decisions to volunteer their responses for points?  Just considering the medium questions, they volunteered more in the hard-start than in the easy-start condition.  We know that people were more confident in the hard-start condition, but perhaps they also set a lower criterion for how confident they had to be in order to volunteer an answer?  No, it doesn’t look like it; Pansky and Goldsmith found no difference in estimated report criterion for the hard-start versus easy-start condition.  So the difference in volunteering rate seems to be driven just by the difference in confidence ratings.

To sum it all up: the difficulty of an initial part of the test (hard-start vs. easy-start) affected participants’ confidence in their subsequent answers, but did not affect the accuracy of those answers, nor the response criterion participants used for deciding to volunteer their answers.

We could think of this as roughly analogous to something like a mental set or framing effect, but for metamemory:  An initial situation sets up an expectation that persists throughout the test, against which we contrast our retrieval attempts on later questions.

This makes me wonder about implications for any test in which question difficulty varies from beginning to end, and in which there is a penalty for incorrect responses.  For example, computerized adaptive tests adjust their difficulty based on the test-taker’s performance, either on an item-by-item schedule (e.g., GMAT) or across sub-sections of the test (e.g., GRE). So for a given test-taker, later questions often end up being easier or harder than the initial questions.  This contrast could skew the test-taker’s confidence judgments, which in turn could influence the effectiveness of his/her decisions to volunteer or withhold each response.  It’s tricky to see how this metamemory effect might differentially play out in terms of the final score for test-takers at different levels of memory ability.  There certainly seems to be room for further research here.

As we come to better understand the factors that influence our metamemory judgments we will learn to better choreograph the perpetual dance between our memory and metamemory.

Time to go and say hi to Fred.