This #time4action special issue of Attention, Perception & Psychophysics is exciting. Joo-Hyun Song and Timothy Welsh have assembled a large and impressive set of articles highlighting the importance of action for understanding cognition. In general, the special issue illustrates how cognition and action (and perception, too) are highly integrated aspects of what we call “the mind.” The depth of that integration, and whether we can separate cognition, action, and perception as separate “systems” or “modules” or whatever, continues to befuddle the field (for an illustration of this continuing debate, see Firestone & Scholl, 2016, and provocative commentaries).

So it is difficult to pick a focal paper upon which to comment.

Should I remark on the exciting possibility that bodily position such as standing alters cognitive processing (Smith et al., 2019)? Or the way subtle eye movements may reveal how our mind is encoding memories (Meghanathan et al., 2019)? How we sequence our actions in a way that betrays our need to reduce cognitive load (VonderHaar et al., 2019)? How about epic reviews and updates of influential ideas such as event coding (Hommel, 2019), the resonant brain (Grossberg, 2019), and extrapolation “momentum” effects in perception, action, and cognition (Hubbard, 2019)?

I was drawn to a paper in the issue that illustrates the continuing importance of simple and rigorous experimentation. Classic experimental design still yields an important theoretical focus that isolates clues to answering these big questions about action and cognition and their integration. The work by Halvorson, Bushinski and Hilverman (2019), like many in the issue, illustrates how elegantly simple experiments remain a solid bedrock for theory testing (cf. Oberauer & Lewandowsky, 2019), even as our tasks and stimuli and datasets become more complex and daunting.

Halvorson and colleagues tested how gestures and speech together facilitate (or interfere) with memory. The reason their paper initially intrigued me is that my collaborators and I are interested in the integrative problem of language and communication (Dale et al., 2016). The capacity to use language and to communicate is incredibly complicated. And despite the frequency of this observation, the many perceptual, cognitive, and motor aspects of language and communication are often studied independently. We do not yet have strong insights into how these processes are weaved together during performance, from the verbal to the nonverbal (where action is taken to reside).

Often when language and communication are studied, scientists invoke the convenience of carving them at rough and approximate joints. The mind does not get this privilege. Our minds have to keep pace with the dynamics of our own actions, those of our conspecifics, and an environment that can be in flux.

One domain where integration has been of critical theoretical importance is in the study of gesture (see review in Goldin-Meadow, 2005). Indeed, a long-standing debate on gesture is whether it is integrated with speech so much as to be a proper part of the same underlying system (see collection in McNeill, 2000). In their paper, Halvorson and colleagues ask how perceiving gesture in another person and producing actions ourselves at the same time may influence our memory for what that person is saying.

How does the mind integrate gestures?

Does it use its own motor system to understand gestures?

For example, as Halvorson and colleagues review, if you see gesture and speech in combination, you are more likely to remember the content of that speech. Gesture, somehow, helps you to remember and learn from what is being said (e.g., among many: Cook, Duffy, & Fenn, 2013; Ianì & Bucciarelli, 2017). A psychological scientist might ask whether you, the listener, are using your own underlying motor system to make this happen.

This would suggest integration is occurring through weaving the perceived gestures into your own motor system, perhaps even simulating them covertly (cf. in brain-imaging studies: Schippers et al., 2010; Schippers & Keysers, 2011; see related debate in Cook, Bird, et al., 2013). It would be economical in various ways to solve the integration problem by weaving perceptions and actions, comprehension and production, together into one well-connected system (cf. Pickering & Garrod, 2013a, 2013b).

This idea naturally recommends an experiment: Have participants watch videos of people speaking and using gesture, then interfere with the motor systems of participants by having them carry out concurrent actions (like tapping a desk). Does this interference of a participant’s motor system impact the memory benefits they get from seeing gesture? Prior work has tested this and found that interference does seem to dampen the memory boost (see details in Ianì & Bucciarelli, 2017). So we might conclude that, indeed, the human mind integrates co-speech gestures by using the same underlying “machinery,” because when distracting that machinery (with a concurrent task), these benefits go away.

Halvorson and colleagues  test whether a basic memory effect may play a role here. They adapt the same rigorous setup from this prior work, use consistent materials, and add a critical condition: Participants carry out the distracting actions during their viewing of the videos and when later recalling what was said in the videos. This means that the context cues (tapping on a table) are now consistent across the whole task―encoding and retrieval contexts are matched. What Halvorson and colleagues find is that by maintaining a consistent context, the boost one gets from gesture is preserved; statistically, in fact, they do not detect a difference between their conditions that include constant “distracting” actions and the comparison in which participants are not instructed to move while viewing or recalling the videos. So, in fact, concurrent actions can help memory so long as they are part of a consistent motor context of a task.

Halvorson and colleagues are careful not to overplay the results. Their discussion carefully balances the general patterns in their results with open questions of interpretation. For example, the general encoding-retrieval interpretation may be too simplistic―it may be that a relative boost is preserved with concurrent actions because these actions are relevant to the task. The manual distraction task of participants is consistent with the manual gestures perceived in stimulus videos. In fact, when inspecting their data visually, it is difficult not to wonder if there may be power issues lurking here, and the best boost to memory may indeed be the undistracted condition, and the present power may have been insufficient to detect it. This is relevant to underlying mechanisms of course, because one cannot really infer equivalence of conditions under a null comparison (only hold their difference in abeyance).

So, despite the simple elegance of this kind of design, more is needed. But this is the beauty of simple, controllable designs. Halvorson and colleagues describe one way forward, and they illustrate it through describing the work of Ping et al. (2014). Here, and in the future directions proposed by Halvorson and colleagues, we could use both the relevance and complexity of concurrent actions as focal ingredients to vary.

Properties of concurrent tasks may be carefully parameterized. Through such parameterization, the psychological scientist can isolate causes of memory effects from accompanying behavior in a listener. It is entirely possible that expanding such experiments like those of Ianì and Bucciarelli (2017), Halvorson and colleagues, and Ping et al. (2014) may help isolate the “integrative mechanisms” of perceiving and producing gestures and their impact on language and cognition through a carefully designed behavioral task alone.

This commentary began with concerns about the importance of integration in our understanding of mind―in particular, that action should not be considered apart from perception and cognition, but instead a dynamic and interacting part of it. In my own area of language and communication, I have been particularly inspired by tracking the dynamics of action in various tasks to understand how complex communicative behaviors are organized.

This #time4action special issue by Song and Welsh offers a wealth of articles that further attest to the importance of action in psychological science. The special issue is evidence that the compelling and heartfelt lamentations of Rosenbaum (2005) about action’s undue neglect have achieved some melioration. In the specific paper I considered, Halvorson and colleagues remind us of the importance of carefully parameterized experiments for isolating what underlies integrative properties of several behaviors and processes: gesture, motor cognition, memory and language. Importantly, in these designs, actions either perceived or produced, are the windows onto this integrative system.