Is this the right assessment for you? Is it right for me?

Imagine you’re back in school (it may have been a while for you!) and you’re undergoing some kind of psychometric assessment, perhaps to see if you need extra support to be the best student you can be. If you’re reading this, it’s probably been a long, long time since anyone sat you down to test your memory capacity in an educational setting – your colleague testing out a new experimental paradigm or assessment on anyone who is willing to sit down doesn’t count! With any assessment, we need to ask whether the assessment is fair; both whether it can measure what we think it measures, but also whether it’s fair to the people who will take the assessment and be scored by it.

This question, and the underlying assumption of differences, is at the core of Tyler McFayden, Maria Gonzalez Aguiar, Charlotte MacKenzie, Anne McIntosh and Kristi Multhaup’s (pictured above) review and meta-analysis of memory differences between Deaf signers and hearing nonsigners published in Psychonomic Bulletin & Review. As they describe in their review, there is a pervasive belief that Deaf signers have impaired memory, implicitly as a result of their lack of hearing. Any such blanket statement should, of course, be taken with a grain (or maybe a bucket) of salt, but rather than just opting for either belief or skepticism, the authors chose to ask the literature what we genuinely know, rather than what we think we know about memory differences between Deaf signers and hearing nonsigners.

To break down the question at the core of their work, we need to talk about what we mean by memory in the context of this review. McFayden and colleagues were particularly interested in determining whether there were differences in short term memory and working memory between their populations of interest. Now, there are lots of different ways to assess both of these, but each assessment will tap in to these processes in distinct ways, and the particular facet each assessment probes is key. Some of the studies in the meta-analysis use verbal serial order tasks, where the person being assessed needs to report the sequence of presented words or numbers either forward or backward, where others use visuospatial serial order tasks, where they would indicate which objects were touched in what sequence. Both of these approaches can tell us about short term memory and working memory, but the underlying question is whether a given person has developed the particular facet of memory that is probed by a particular assessment.

To understand what the literature on this topic is capable of telling us, the authors used the PRISMA selection procedure to search for papers with keywords which indicated they might speak to the question of memory differences between Deaf signers and hearing non-signers. In the end, this process of filtering gave them a final sample of 32 papers for their metanalysis. What they found has substantial implications for the scenario we started with – the only evidence they found in support of memory differences between Deaf signers and hearing nonsigners was when Deaf signers were assessed with verbal serial order tasks. The key thing to realize here is that the Deaf signers simply use memory differently – the meta-analysis found no differences between the groups when memory was assessed with visuospatial serial order tasks.

Figures 2 (top) and 3 (bottom) from their paper, showing differences for verbal forward recall (assessing short term memory) and verbal backward recall (assessing working memory)

So, what should we all take away from this paper? The implications go beyond the immediately obvious, that we should assess people with appropriate tools, and speak to something deeper in psychology. While the body of psychological research and theory argues very convincingly that we all have essentially the same cognitive processes, there’s also a critical need to ask how a given person has developed a particular capability (or not) and whether that is a reasonable assumption to make. If we think about the populations in this paper, it’s probably unreasonable to think that Deaf signers would have a need to develop the particular facets of sequential short-term and working memory that a verbal serial order task probes, therefore using these tasks simply doesn’t make sense in this context. There’s a lesson here for all of us: we need to ask why our populations should (or shouldn’t) have a capability we think they do, and if we ask this question first, we’ll not only do better science, but we’ll gain a deeper understanding of the varying workings of the human mind.

Featured Psychonomic Society article

McFayden, T. C., Gonzalez Aguiar, M. K., MacKenzie, C. C., McIntosh, A., & Multhaup, K. S. (2023). Verbal and visual serial-order memory in deaf signers and hearing nonsigners: A systematic review and meta-analysis. Psychonomic Bulletin & Review, 1-18. https://doi.org/10.3758/s13423-023-02282-6