In this episode of All Things Cognition, I interviewed Michelle Rivers and Steven Pan (pictured below) about their recent Psychonomic Society journal Memory & Cognition paper called “Metacognitive Awareness of the Pretesting Effect Improves with Self-Regulation Support.” In addition to talking about the benefits of pretesting (or prequestions) on learning, we also talked about other effective ways to learn and how to help motivate learners to appreciate these methods.

Transcription

Intro

Caballero: You’re listening to All Things Cognition, a Psychonomic Society podcast. Now here is your host, Laura Mickes.

Interview

Mickes: Today, I’m talking with Michelle Rivers and Steven Pan. Thanks for joining me.

Rivers: Thanks for having us.

Pan: Thank you.

Mickes: I know Michelle because of the work that we do as members of the Psychonomic Society’s digital team, and I’ve known Steven for years. We took several undergraduate classes together many, many, many years ago. But today we’re talking about a paper that they co-authored and published in Memory & Cognition called “Metacognitive Awareness of the Pretesting Effect Improves with Self-Regulation Support.” So I guess a good start would be describing the pretesting effect. What is that?

Pan: So the pretesting effect is the memory benefit of taking practice tests on information that you have yet to learn, followed by studying the correct answers. So an example of that would be, say I asked you what is the capital of Australia, and perhaps you haven’t studied that before. You might guess Sydney or Melbourne. And then I tell you that the correct answer is Canberra. So you’ve just engaged in a pretest, a practice test on information you have yet to learn and then study[ing] the correct answer or being told the correct answer. And a typical finding is that engaging in that form of pretesting improves memory relative to simply studying the correct answer from the very beginning. So say in a read condition, you were told, and you just read the capital of Canada is Ottawa. So that’s one way of learning information. Just read it; alternatively, you can guess what the correct answer is, then be told what the correct answer is, which is pretesting.

Mickes: Right.

Pan: And that typically results in better long-term memory than reading.

Mickes: Okay. And there’s not awareness about it. So people don’t know that this actually can benefit learning. That’s right. According to the, the title of your paper.

Pan: Yes, that’s correct.

Mickes: Why did you do the research? Can you give me some background?

Rivers: I think it’s kind of counterintuitive if you think about it, the, the pretesting effect in general because you’re producing an erroneous response when you’re first asked that question. Right. So in fact, we’ll eliminate people who, who do produce a correct answer. So we actually are hoping that people produce an incorrect response and then correcting it by giving them that corrective feedback. So I think part of the, the reason why people might not appreciate the benefit of pretesting is that they’re producing an error. So they might think, well, if I’m producing an error, maybe I’m gonna remember the error. And there’s this idea in learning that maybe we shouldn’t be making mistakes, that mistakes are a bad thing

Mickes: Yeah.

Rivers: So that could be part of the reason that students don’t appreciate the benefits of pretesting.

Mickes: I think I would think that too. And in fact, this is a fairly new finding, isn’t it? That people, people aren’t that aware of this prequestion effect that actually boosts memory. What were your hypotheses going in?

Pan: We had really two main hypotheses. The first one was that participants or learners engage in pretesting and reading, and they may have more than one experience doing that, [but] even with that experience, they would not necessarily recognize that pretesting is better for learning. And the second hypothesis was that perhaps we could then give other forms of support, provide learners with information that would help them develop that awareness that pretesting is actually good for their learning.

Mickes: How did you go about testing those hypotheses?

Pan: Well, we conducted a series of experiments and they all had a common design. So each experiment involved participants going through multiple, what we call cycles.

Mickes: Okay.

Pan: And within each cycle, they would learn a set of materials, paired associates, word pairs, and they would learn them through pretesting and reading. Some word pairs are pretested, some words are read. And after they’ve learned that information, we would ask them to predict how well they would perform on a subsequent memory test for the items that had been pretested and the items that had been read. And so that’s sort of our index of whether they are aware of the benefits of pretesting on memory. And then after a five-minute delay, they would take that memory test in which the typical finding is that the items that were pretested will be better remembered on that memory test. So that’s what happens in a single cycle.

Mickes: Okay.

Pan: And in each experiment, we’d have participants go through at least two cycles. And the key question was, after repeated experience, whether they would improve their predictions to more accurately reflect the benefit of pretesting for learning. And then across experiments besides what I just described, after the memory test, the first memory test, we would give them certain forms of external support, different types of information on, for example, on how well they performed, which could potentially help them develop greater appreciation for the benefits of pretesting. So that’s overall across all these experiments, that’s the basic design.

Mickes: Okay. And you had five different experiments? Yes. Each of them tweaking this question or trying, trying to get people to appreciate that prequestions will benefit them.

Pan: Right? Yes. And so across the five experiments, the first four, we progressively increased the amount of external support we gave our participants. So in the first experiment, we didn’t give them any additional information, we just had them go through two cycles, and we wanted to determine whether experience alone would be sufficient to develop, to foster that metacognitive awareness. And then experiments two, three, and four, we gave more and more information. So in experiment two, after the first memory test, we told participants how well they did on the items that they had learned through pretesting and the items they had learned through reading. So we gave them performance feedback. [Procedure illustrated below.]

Mickes: In general?

Pan: Overall. So we would tell them, for example, you learn 16 paired associates through pretesting and you scored on this memory test, say you recalled 10 out of 16 correctly. And then we would also tell them you learned another 16 pairs through reading and you scored maybe 5 out of 16. And so that’s the feedback we gave them in experiment two,

Mickes: And then you had them do the other cycle to see if—

Pan: Yes. And then we wanted to see whether experience plus that external support in the form of feedback would result in increased awareness of the benefits of pretesting.

Rivers: I think it’s kind of striking even after the first two experiments In experiment one where we just had experience alone. So participants take this test, they are experiencing the pretesting effect themselves. So if they realize that they should use that to inform their subsequent predictions on that second cycle. Like, ‘oh, wow, I really did better on these pretested items.

Mickes: Yeah,

Rivers: Amazing, I didn’t realize that. Now I should use that to inform my subsequent prediction.’ And they don’t. So this is kind of like the idea of maybe experience isn’t the best teacher, maybe they need some additional support.

Mickes: Right.

Rivers: And so that was what informed our subsequent experiments. So we were thinking, okay, well, if experience alone doesn’t work, what other supports can we provide our student participants?

Mickes: I see.

Rivers: Mm-Hmm. <affirmative>.

Mickes: So what was the idea then? What did you do next to provide more support?

Pan: So we first gave them, as I mentioned in experiment two, feedback on how well they did. [Feedback shown below.]

Mickes: Yes.

Pan: Then in experiment three, we added on top of that reminders of how they had predicted their performance in the first cycle. So we told them, see, this is how well you did, and look, this is what you predicted. So we can cause learners to contrast their predictions with their actual performance.

Mickes: Oh, I see.

Pan: And then in experiment four, in the second cycle of that experiment, we asked participants right before they made their predictions to recall how well they did in the, on the memory test in the first cycle, sort of another way of reminding them of the benefits of pretesting.

Mickes: Right. That’s clever.

Pan: From experiments one through four, increasing our external support.

Mickes: Right.

Pan: And then in experiment five, we went back and we compared feedback alone in terms of performance versus feedback with reminders of predictions together.

Mickes: And what did you find in experiment three?

Pan: So experiment three was the first of, in our series of experiments where we had a statistically significant evidence of updating of predictions to reflect the benefits of pretesting.

Mickes: Mm-hmm. <Affirmative>. So,

Pan: So in the first two experiments from cycle one to cycle two, participants’ predictions did not reflect the benefits of pretesting. They continued to believe that they would perform about equally well on pretested items or on items they had read. But then in experiment three, that was the first instance where in the second cycle, their predictions for pretesting were significantly higher than for reading.

Mickes: Really? Remind me again what you gave them in experiment three that was extra from experiment two.

Pan: So in addition to their performance and how on the final memory test in the first cycle, we also reminded them of what they had predicted their memory test performance would be.

Mickes: And that did it?

Pan: In experiment three, that that was the first instance of a successful awareness of pretesting.

Mickes: That’s incredible. First of all, I’m surprised that they didn’t get it from experience. I find that very surprising. But it’s really cool that, all right, we give them or let’s give them a little bit more information, can we get them to realize that this pretesting actually works? And you did. And what about then for experiments four and five?

Pan: So for experiment four, when we added recall of test performance, so in second cycle, they had to remember how they had done on the memory test. In the first cycle, the updating in terms of raw magnitude was even larger. [Experiment 4 predictions and performance results are shown below.]

Mickes: Oh.

Pan: But to be clear, in experiments 1, 2, 3, and 4, we are testing slightly different, or sometimes more than slightly different ways of fostering this awareness of pretesting. We are not directly, up to that point, directly comparing any of these methods.

Mickes: I see.

Pan: And so in experiment five, that’s where we directly compared two forms of external support performance feedback alone versus feedback with reminders of how participants had predicted their performance to be. And, and in that final experiment, we found that both methods were, statistically speaking, equally effective at fostering…

Mickes: Really?

Pan: …awareness of the benefits of pretesting. So either you tell the participants how well they did on the memory test…

Mickes: Mmm.

Pan: …or you tell them how well they did, plus remind them of how they had predicted their performance to be; both of those methods foster awareness of the benefits of pretesting.

Mickes: Now, why is this beneficial? This is a, maybe a stupid question, but what if there wasn’t metacognitive awareness and say teachers and professors just use it <laugh> and, and unbeknownst to the students where they’re, they’re answering these prequestions and it will benefit them. Why does the learner need to know that it’ll benefit them?

Rivers: This is a question that motivates a lot of my research as a self-regulated learner slash metacognition focused researcher and kind of the intersection of Steven and I’s interest here, it seems that learners are unlikely to adopt strategies if they don’t believe that they’re going work. So really the idea here is we want to get students to use these strategies when they study on their own.

Mickes: Yes.

Rivers: And similarly teachers, if they aren’t aware of effective methods for learning, they might not be instantiating them or they might not instantiate them in the most effective way. So this is kind of a, a first step at getting students to recognize the benefit of a particular strategy. Obviously it’s not the only barrier that students face when it comes to regulating their own learning. There’s also issues like managing their time effectively and staying motivated, that’s also going to affect what strategies they adopt,

Mickes: Mm-hmm. <Affirmative>

Rivers: But having accurate knowledge and beliefs about the strategies, especially the belief that the strategy works for them, not just for learners at large,

Mickes: Right.

Rivers: Again, is just kind of the foundation for promoting effective self-regulated learning.

Mickes: I see. How would you recommend a student then who’s studying on their own implement this?

Pan: Well, there are some very simple ways to engage in pretesting. One way is if you have access to practice test questions, for example, in a book chapter or provided by instructor, you can start using them, for example, in the advance of a, a lecture or a lesson, you can simply test yourself on information you have yet to learn. And that gives you a little bit of a preview perhaps of what you’re going to encounter. And it is likely to foster a pretesting effect because then you will learn the correct answers and you will probably learn those more effectively than if someone just told you the correct answers from the very beginning. So that’s one way.

Mickes: Yes. How do you recommend then to teachers and professors, any educator, for, for us to use it? Do we just go in into a lecture and start off by asking questions about what’s to come? Do you recommend we do that?

Pan: I’d say there is accumulating evidence that doing that at the, for example, starting off a lecture with a set of thought-provoking questions to get students thinking about the material they’re about to encounter, I think that could be a profitable strategy.

Mickes: Yeah.

Pan: At the same time, we don’t have a great many studies of pretesting or prequestions in authentic education environments, so the exact nature of how to implement it for the best results has yet to be determined.

Mickes: We don’t. Okay. We don’t know yet.

Pan: Yes.

Mickes: Do you use it? Do you ask prequestions when you teach? Michelle, are you teaching?

Rivers: Yeah, I do. I teach. I do kind of a combination of prequestions and post-questions, which we would just call retrieval practice or practice testing. I kind of lump these two strategies together. Basically, any type of strategy that involves active recall or retrieval is going to be beneficial as long as there’s correct answer feedback afterwards. So using a combination of testing before a lecture, after a lecture, before reading a textbook, after reading a textbook and spacing that out across time is going to lead to benefits, especially in comparison to some of these more passive strategies that students use. Like simply reading or rereading or highlighting or just looking at their notes. Right. So mm-hmm.

Mickes: Right. I talked about applying what you know before I asked you: Why do you think that the prequestions work to improve memory?

Pan: Well, that’s perhaps the million-dollar question that we’re all still trying to figure out. But I think there’s a growing consensus that prequestions change subsequent study behavior. So, or pretesting or prequestions (the terms are sort of interchangeable). When you give somebody a pretest and they attempt to answer that question and usually answer incorrectly, then you show them the correct information. They might pay closer attention to it. They might be curious about the answer.

Mickes: Right.

Pan: They might, as they are studying the correct answer, be thinking about their previous guess.

Mickes: Yeah.

Pan: And maybe doing some sort of comparison between what they thought the answer would be versus what it actually is. They might also be given sort of a, a bit of a surprise as to how much they actually do or do not know about a given topic. And so that changes how they learn. And there are hosts of possible…

Mickes: Right.

Pan: …activities that can occur after you’ve taken a pretest that change how well you learn information. So the question though, of course, is which of those, or which combination of those actually are responsible for the pretesting effect? Possibly several mechanisms are at play.

Mickes: And is this something that you’re investigating now?

Pan: Yes. From a variety of different angles. So for instance in my lab we have some evidence that taking a pretest changes attention; literally causes learners to focus more. We’ve also had some data where we compared, for example, pretesting with retrieval practice, you know, taking a test before you’ve learned the information versus after you learned the information. And pretesting is sometimes as effective or more effective than that strategy, which is quite surprising.

Mickes: Really?

Pan: So then there’s a whole host of questions as to the mechanisms involved in memory retrieval of something you’ve already learned versus something you haven’t learned yet (and then you learn it). There’s much more to come in terms of trying to figure out why pretesting is as effective as it is, as well as the circumstances under which pretesting is not as effective. So there are some data where, for instance, if you’re pretesting people on information they have absolutely no clue about and it’s very complicated, it doesn’t seem to be as effective versus pretesting on something where they might have some intuition [but] might not be correct about what the answer might be. Of course, all this is very tentative. The pretesting literature is still in some aspects quite, quite in its infancy.

Mickes: Right. It’s early and there’s so much to do. It’s exciting. It potentially changes strategies during encoding. That’s what it sounds like. People are paying more attention then to the things that they were asked about that they didn’t know than that helps them hone their attention or hone their focus on those items. That seems to be beneficial. But Michelle mentioned something about retrieval practice. Does that play a role here?

Rivers: It could. Another possibility, along with the ones that Steven mentioned, is just that you’re accessing a bank of knowledge that then creates a nice foundation for new knowledge to stick to. Right. So you’re activating prior knowledge and the new knowledge can link onto that.

Mickes: I see.

Rivers: Mm-Hmm. <affirmative>.

Mickes: Okay.

Rivers: Another possibility, kind of expanding on what Steven was saying is that the incorrect answer that’s provided during pretesting could then be like a mediator to get you to the correct answer. So saying the capital of Australia is Sydney. Oh, it’s not Sydney, but I remember that I said it was Sydney and now that’s going to make me remember that it’s actually Canberra.

Mickes: Right. So that’s not…

Rivers: Yeah. And well, there could also be that element of surprise or emotion that plays into that. Like, oh, I, I really thought it was this answer, but it’s not. And now that, that helps me remember even more.

Pan: And also in addition to surprise, possibly curiosity in terms of.

Rivers: Mm-hmm. <Affirmative>

Pan: Oh, I, I really now want to know what that correct answer actually is.

Mickes: Right.

Pan: Or know whether my guess was right or wrong.

Mickes: Yes. Do they have to be guesses? You mentioned, I really thought the answer was do you ever collect confidence in their knowledge? So you ask them, what’s the capital of Australia? And you know, I could say 90% sure it’s Sydney. Do you collect confidence and does that matter how confident they are in their initial answer versus how much they’ll remember later?

Rivers: So this is a really great question, Laura. You’re thinking about an effect called hyper-correction that it’s related to pretesting. I have not done any research on this effect, but I think it’s super fascinating. So exactly what you were thinking, which is, and it is also counterintuitive, but high confidence errors are, and then corrected high confidence errors are more likely to be remembered than low confidence errors.

Mickes: Wow.

Rivers: So for example, if I’m very confident that the capital of Australia is Sydney and then I’m told it’s Canberra, I’m more likely to remember Canberra, which again is, is counterintuitive cause you would think, well if it’s a high confidence error that’s pretty highly ingrained knowledge.

Mickes: Right.

Rivers: But there may be that surprise factor that helps you remember then that the correct answer is actually Canberra. So this could be useful in say correcting misconceptions that students have. Again, not viewing students as this blank slate when they come into the classroom, but they, they carry this knowledge with them and some of the knowledge that they come into the classroom with is wrong.

Mickes: Yeah.

Rivers: Right. And probably held with high confidence, right? Like that we have different learning styles or that there’s two sides of the brain and they work differently and these types of things. Exactly. And instead of just correcting those, maybe actually priming them and asking them to produce that knowledge first and then correcting it so that they get that benefit of pretesting and hyper correction and all these, all these related effects that are at play in the classroom.

Mickes: What’s next?

Pan: Well we’re just, I would say scratching the surface of what could be done. But one particular angle I think is a very logical follow up to this work is to explore it in a more authentic educational setting. So the experiments in this particular paper were conducted online with, you know, paired associate materials, not exactly the kinds of information that students actually typically try to learn. So perhaps we should investigate that with for example, text passages or other kinds of stimuli that are actually learned in, for example, classroom environments. So that’s one direction.

Mickes: Mm-Hmm. <affirmative>.

Pan: Beyond that the question as we’ve sort of alluded to in terms of what are the theoretical mechanisms at play for the pretesting effect, there’s ongoing work on that in my lab and in other labs and plenty more beyond that.

Mickes: Right. And Michelle, are you doing more on the self-regulated learning angle with prequestions?

Rivers: Yeah, exactly. So like I mentioned, there are a lot of barriers to students adopting and using effective strategies like pretesting, like retrieval practice, like spacing out your studying and knowledge about the strategy and believing that it works is probably the most foundational, but it’s obviously not the whole picture. I’ve been looking at some of these other barriers to student success, like time management and motivation. So getting students to actually tie the use of these strategies to their long-term goals. And then also making a plan to use them. Because it can be difficult to use these effective strategies when all you’re doing is cramming for each exam as it comes on your schedule. If you space out studying across time, you have more opportunities to use strategies such as pretesting and post-testing or retrieval practice.

Mickes: I see. Now do you have any success in getting students to appreciate these better strategies for learning? Because I don’t know that I have. I tell them test yourself. I tell them all the time, just don’t highlight, don’t just read you’ve, you’ve gotta test yourselves. And I don’t know that they actually do it and I also say don’t cram, but I don’t think they listen to that <laugh> Maybe the students are doing it, they just don’t tell me.

Rivers: Yes. It’s a huge challenge. So I’ll say I have more studies in which there are no effects than there are effects. So yes, I’ll acknowledge that it’s very difficult to change student behavior, especially when it’s so ingrained. I mean students are rewarded, unfortunately, for cramming. You can perform well on an exam if you cram. The problem is you don’t remember the information long term. So if your goal is only to perform well on an exam in the next few hours, then might as well cram and and study as much as you can in that moment.

Mickes: Which is why you made it a point to say it’s gotta fit into their long-term goals.

Rivers: Exactly.

Mickes: It can’t just be for the next test.

Rivers: Right, right. And then some of the issue with just direct instructions saying, hey, you should use this is students are very good at saying, ‘okay, I know that you know, Dr. Mickes said to do this, but this works for me.

Mickes: Yeah.

Rivers: You know, everyone else should use practice testing.

Mickes: Yeah.

Rivers: But I benefit from re-reading.

Mickes: Okay.

Pan: I’ll just add that in some cases, one of the prime opportunities at least that I personally have found to encourage the use of effective learning strategies is after students have had the opportunity to try their existing methods and then perhaps after the first, say, midterm exam in a course and then you have students coming to office hours saying, well I, what can I do to improve? And that is often a really great time to then sit them down and ask them, so what did you do for this exam? How did you prepare? And then you’ll often discover that they’ve used a whole host of very popular but highly ineffective strategies. For example, they copy their notes multiple times, they highlighted it in different colors.

Mickes: Yeah.

Pan: And so on. They spend hours and hours doing that. And then you can tell them, you know, this is a better strategy, a better set of strategies. It might take you actually less time but you’ll get better results. And in my own teaching I’ve had multiple cases where the students will come back after the next exam, they have completely changed their strategies and they’ve gotten a full letter grade higher sometimes even more beyond that. So, well there are cases where it can be quite effective, but sometimes you have to let the students try their own methods and, and get the results that might be common with those methods and then give them the opportunity to do something different.

Mickes: Yeah, that’s a really good tip for us. Now tell me when you were students, cause obviously, well Steven and I knew each other as undergraduates <laugh>, so we knew what kind of students we were, you know, very serious. I could guess Michelle was too. What did you use as an undergrad? Cause I used it all. I highlighted, I reread, I recopied my notes, I tested myself, everything. I just spent too much time studying <laugh>

Rivers: <Laugh>. Yeah, me too. <Laugh> I think I used, and this is consistent with the survey research that is out there, I used a combination of both effective and ineffective strategies.

Mickes: Okay.

Rivers: So, part of that could have been kind of relying on these less effective strategies like rereading and recopying notes because it was like a security blanket.

Mickes: Yeah.

Rivers: It made me feel like, okay. You know, it, it contributes to that sense of fluency, like you know the material well. Unfortunately, that sense of fluency increases your confidence but might not actually improve your memory. Whereas strategies like practice testing are, are really helping your confidence and your memory as you’re realizing what you know and what you don’t know. And you’re also getting that boost for memory and comprehension later on down the road. So I think I did a combination of both. If I were to do it all over again, they would use even more practice testing and less of those less effective strategies.

Mickes: I, yes, I think I would too. Steven?

Pan: Well, I remember, I think my first year as an undergrad, I took a child development course and the first lesson the professor made a point of telling her opinion on what was the best way to study. And she said, look at your calendar. You have now until the exam, you should plan out study sessions where you repeatedly view information, and you don’t wait till the last minute…

Mickes: Good!

Pan: …to study. And that was a great reminder because from that point onwards, I really tried to follow that and tried to, what I now know is use distributed practice to revisit information and avoid cramming. And so it became sort of this drive to avoid waiting to the last minute to study for the exams and instead to really have good preparation. Ideally, the week before the exam you’ve already studied most of the information and so the, that final week is sort of a bonus where you can just catch up on anything that was a little left over.

Mickes: Right.

Pan: Whenever I was successful in a course, typically I believe that strategy was used.

Mickes: Mmm.

Pan: And then also I found myself practice testing a lot.

Mickes: Yes.

Pan: I remember making notes and then quizzing myself with the notes. Right. And I didn’t know at the time that that was a retrieval practice strategy that would be highly, highly effective. But I’m so glad that I did use that quite a bit and.

Mickes: Yes.

Mickes: Yeah. When I found out about retrieval practice and how successful it can be at helping increase learning, I, I was so proud of myself for stumbling on it when I was an undergrad. I, I guess maybe some of us just do stumble on it, but it’s really excellent that you had professors telling you, okay, this is how you should do it. And I like that she said, get your calendar out and figure it out; the times that you’re going to study, I think Michelle mentioned something about long-term. So cramming is good for you, cram right before the test, you take the test then you’re going to forget everything. But what about the prequestion effect and long-term effects? Has there been work done on that? Will they revert back to thinking that the capital of Australia, Sydney in a week, in two weeks?

Pan: In terms of reverting back to the original incorrect response? I’m not sure of the particular data, although I’m, I would imagine there is a study somewhere that would directly speak to that issue. But more broadly the longevity of the pretesting or prequestion effect, that was a matter of debate even a couple years ago because most of the pretesting studies, the retention interval was extremely short. It was an immediate test or five minutes later. But in recent years there have been several studies, including some from my own lab, which have shown that the pretesting effect persists across days, weeks and so on. So you will get those benefits over a somewhat longer period of time than in the typical pretesting study.

Mickes: Mm-hmm. <Affirmative>.

Pan: And so I would say that there is at least some positive indications that the pretesting effect is beneficial for, I might say is meaningful long-term memory.

Mickes: Right. Okay. Is there anything you’d like to add that I didn’t ask?

Pan: Well, one thing I think is really fascinating about pretesting is it really goes against sort an idea that we touched upon earlier, which is the value of making errors, guessing something incorrectly, making a mistake, having a wrong idea about something. And there’s some data from my lab and elsewhere showing that people generally don’t think that is valuable except for learning to avoid something that was done incorrectly. You might learn that, oh, this is wrong, then I won’t do it again or I won’t say that again or I will correct that mistake. What pretesting shows is that sometimes you guess incorrectly, that in of itself that active guessing incorrectly can lead to better learning if you then get to study the correct answer afterwards. So this is a different twist on the value of making an error.

Mickes: Right.

Pan: And it goes to really show that perhaps we have undervalued making mistakes, making errors because this is yet another example of the value of those errors.

Mickes: Right. That’s really interesting. I do a lot of research and eyewitness identification, making errors there could cost someone’s someone’s life. So we try to stay away from errors as much as we can, but I see, I see there would be real value in coming up with the wrong answer because at least you’re coming up with something instead of just a

Pan: Yeah. So just to be clear, this is not an eyewitness testimony scenario where your mistaken identification is going to be really, really serious. This is practice testing where your errors have no negative consequence other than perhaps you might be a little embarrassed that you, you guessed something wrong. So errors in that context. Yes, of course.

Mickes: Right.

Pan: If you were a surgeon or you were on a witness stand or you were a pilot. No errors under any circumstance. <Laugh> in most cases, but in this practice testing scenario, errors can be a good thing.

Mickes: And it’s critical that feedback’s given.

Pan: Yes. Errors that are corrected. Yes.

Mickes: Yes. Okay. I really appreciate you talking to me about this research. Not just your paper in Memory & Cognition. That was fascinating in its own right. But just in general, this research on retrieval practice and prequestions, I think I will keep trying to help students be better learners and I think that this has helped me do that. So thank you. Thank you so much for talking to me today.

Pan: Thank you.

Rivers: It’s a pleasure. We love talking about learning research and thinking about this stuff, especially in applied settings like in the classroom. Because we are all educators as well.

Concluding statement

Caballero: Thank you for listening to All Things Cognition, a Psychonomic Society podcast. If you liked this episode, don’t forget to follow the channel using your favorite podcast player or app. All members of the Psychonomic Society receive free access to our seven journals and are invited to attend our annual conference at no charge. Learn more and become a member by visiting us online at http://www.psychonomic.org.

Featured Psychonomic Society article

Pan, S. C. & Rivers, M. L. (2023). Metacognitive awareness of the pretesting effect improves with self-regulation support. Memory & Cognition. https://doi.org/10.3758/s13421-022-01392-1