Mask wearing can alter fundamental cognitive networks for the McGurk effect

As a cognitive psychology professor, I have always loved showing students the never-failing McGurk effect in perception class. After all, who isn’t intrigued by how reliable this effect is? No matter how much you know about it and how many times you have seen or heard the stimuli before, it works.

The McGurk effect is the experience of hearing the sound “da” when in fact you’re listening to the sound “ba” while watching a person’s mouth pronounce “ga”. Check out this video of the McGurk effect:

To experience the McGurk effect, you will need to integrate both visual and auditory cues simultaneously. So, a very good question now as we live through the current COVID-19 pandemic is how mask wearing behavior might affect people’s experience with this effect. Presumably, because we see people’s mouth movement much less due to masks, there could be a loss in visual speech cues in language comprehension in general. Furthermore, would young and older adults adapt differently to this change? And how about gender differences?

Kateřina Chládková, Václav Jonáš Podlipský, Natalia Nuďga, Šárka Šimáčková (pictured below) from the Speakin Lab examined these questions in the Czech Republic and reported their findings in an article recently published in Psychonomic Society journal, Psychonomic Bulletin & Review. The Czech Republic was one of the first countries in the world to institute a mask regulation in March 2020. The authors used this unique opportunity for a natural-setting experiment to study how auditory and visual information interact during the perception of speech.

They first tested young and older adults’ experience with the McGurk effect right after the mask regulation started in late March 2020 and then conducted a follow-up with the participants one month afterward. Within that one month, the participants followed the mask regulation closely — they wore masks in all public places except for their own homes. TV reporters, public figures, and politicians all appeared in the media with masks on. Therefore, the Czech population was consistently exposed to others who spoke with their mouth and nose covered. The authors’ depiction of how mask wearing may affect speech sound perception is in the sketch below.

*Authors’ sketch. How wearing a mask affects speech sound perception.*

All participants were shown a video of the McGurk effect and were giving a choice of 6 buttons to indicate the sound that they heard. The 6 buttons indicate the sounds “ba”, “da”, “ga”, “pa”, “ta”, and “ka”.

The results were fascinating! At the second testing phase, young adults became less reliant on visual information than the first phase; while older adults showed a higher reliance on visual cues (as shown in the figure below).

Chladkova Fig 3 results — *Reliance on a) visual-only perception, b) integrated perception, c) auditory-only perception*

This finding suggests that young adults may be adapting more efficiently than older adults to the loss of visual information due to mask use. A gender difference was also found: women were better lipreaders than men, but men showed more visual-auditory integration of information overall.

These fascinating findings demonstrate the plasticity nature of the human brain. It is mind-boggling to realize how quickly our habits have changed during this past year as we collectively deal with the COVID-19 pandemic. Work, school, entertainment, connecting with friends and family have all changed in drastic ways in the past 12 months. All of these behavioral changes are accompanied by cognitive and neural changes. However, it is still simply astonishing to realize that the McGurk effect, a well-established, fundamental cognitive process that is involved in native-language speech perception could be significantly altered within merely four weeks!

The good news is, in situations where visual information should not be obscured, such as interpreting for and conversing with the deaf population, we can always rely on transparent masks!

Featured Psychonomic Society article

Chládková, K., Podlipský, V.J., Nudga, N., & Šimáčková, Š (2020). The McGurk effect in the time of pandemic: Age-dependent adaptation to an environmental loss of visual speech cues. Psychonomic Bulletin & Review. https://doi.org/10.3758/s13423-020-01852-2

Author

Christie Chung

Dr. Christie Chung is the Esther Lee Mirmow Chair Professor of Psychology at Mills College, CA, USA. Her main research interest is in emotional memory and aging, with a specific focus on the cross-cultural application of the Positivity Effect in memory. Dr. Chung directs the Mills Cognition Lab, where undergraduate students have the opportunity to conduct research studies that explore diverse factors that affect memory, e.g., age, culture, gender identity, and political beliefs. Dr. Chung received her Honours B.Sc. degree from the University of Toronto, her M.A. and Ph.D. from Claremont Graduate University, and her postdoctoral training at Massachusetts Institute of Technology.
View all posts