Once upon a time, in the realm of psychology, a haughty woman summoned her two beloved daughters – perception and attention – and said them: “Tonight the Prince of higher cognition will give a ball. All persons of fashion are invited – including you, my darlings.”
Her stepdaughter – action – was listening, too. But she was not expecting to be invited to the ball. She is only used to executing the capricious orders of her mother and sisters.
A central question of this #time4action issue of the Psychonomic Society’s journal Attention, Perception, & Psychophysics is whether it’s time for action―the Cinderella of psychology―to join the higher cognition ball. By reviewing “a growing amount of empirical evidence showing the relation between action and visual awareness” and “perceptual confidence”, the paper by Anzulewicz et al. foresees a happy ending (as in the original novel); and that action may have intimate relations with higher cognition. Anzulewicz and colleagues suggest multiple reasons why action may influence decision-making and metacognition: for example, because it may count as evidence during an evidence accumulation process, restrict the number of alternatives, or modify stimuli perception.
To better understand why these (and other) reasons may all make sense, but were so far largely neglected, it is important to briefly summarize the long and difficult journey that action (like Cinderella) has traveled in the attempt to join the ball.
It all started with traditional serial models of cognition, which describe perception, decision and action as sequential (and largely independent) processes, with action relegated to the role of executing decisions. This serial account of behavior is far from being out of fashion; it is implemented (more or less literally) in dominant computational accounts of decision-making based on evidence accumulation, which assume that action starts once the decision process is completed (i.e., after evidence crosses a threshold).
Yet, serial models would be quite limited outside the lab. Animals chasing prey or goalkeepers trying to intercept a ball often need to start moving before they could collect enough evidence to make an informed decision, while also remaining sensitive to subsequent information to revise their plans along the way. To account for this, a second class of models―continuous flow models―has been developed, in which a decision module continuously feeds the intermediate state of its computations (e.g., which choice option received more supporting evidence so far) into an action module, so that action can start before the decision is completed (and the threshold crossed). However, even in continuous-flow models, action cannot join the higher cognition ball. Action runs in parallel to decision processes, without influencing them.
One can thus consider a third class of models―embodied models―in which action influences (or feeds back on) decision processes. These models are only now beginning to be investigated. One possibility is that decisions are not made within a central and modularized process, but through a distributed consensus: different aspects of the decision may be resolved in parallel by different brain areas, which mutually influence each other until some consensus emerges. Some aspects of the decision, such as those regarding affordances and motor costs of alternative action plans, may be resolved within action-related systems in the brain, which would therefore participate in the decision rather than just report it.
One can gather more insights on embodied decisions by using experimental setups that retain situated and dynamical aspects of the choice. For example, one can ask participants to use a computer mouse to reach and click one of two response buttons, and then look at how the mouse trajectories unfold in time and space. Despite its simplicity, this setup retains some elements of embodied choices “in the wild”, such as those faced by predators that have to approach prey. Analyzing curvature and velocity of trajectories over time during the choice permits inferring the momentary belief state of participants (e.g., their uncertainty), and it allows us to look more closely at the interplay between decision and action dynamics.
Experiments using this setup reveal several findings that are unexpected from the perspective of serial and even continuous flow models.
First, in most cases participants start moving rapidly, but not necessarily towards the button that they will finally select. Often, their initial direction of movement is halfway between the two buttons. In other cases, participants move initially towards one button and then change their mind abruptly, to reach the other button. These findings reveal a more complex interplay between decision and action than imagined in serial models.
But there is more than that: action dynamics and motor costs influence not just the trajectory but also the final choice. For example, when the two response buttons are far away from one another, making changing one’s mind more costly, participants change their mind less often. If during a perceptual decision, one of the two options entails a smaller motor cost, you tend to select it more often when uncertain. One can model these results in terms of a decision process that simultaneously optimizes both the accuracy of the decision and the cost of the decision-associated action. This raises the question of how the brain computes these costs.
If motor costs were fixed, cost-benefit decisions could be done centrally, before starting an action. But during situated decisions, costs need to be continuously re-evaluated as they change constantly during the unfolding of the action, as a function of the changing geometry of the environment (e.g., hand-ro-target or predator-to-prey distance). We need to continuously adapt to this changing decision and affordance surface; and it makes sense, for example, to change one’s mind less and less often, the more we approach a button (or a prey), as the associated costs grow when hand-target (or predator-prey) distance decreases.
From a more functional perspective, embodied choices like “what button should I click?” may use multiple sources of evidence, from sensory-based (“am I seeing stimulus A or B?”), to prediction-based (“do I predict A or B?”) or action-based (“am I moving towards the left or right?”) and value-based (“is clicking A more valuable or costly than clicking B”?). This suggests that standard evidence accumulation models should be extended to also include other factors including actions (or corollary discharges) as evidence. This view is related but subtly different from the idea that when making two consecutive but related choices, the first decision (press A) can be used as evidence for the second decision (confirming A or changing), hence making it more conservative. Embodied theories of decision would predict that actions (executed or perhaps just planned) influence current and subsequent decisions, above and beyond the effect of previous choices.
Interestingly, the predictions of embodied theories of decision extend nicely to metacognition. Let’s assume that initiating an action towards one of the response buttons (or just planning it) creates commitment and gathers evidence for the button-related choice. If confidence were conceptualized in statistical terms, as the probability that a choice is correct, then action dynamics may influence choice confidence, by affecting the probability calculations. This would explain action effects on metacognitive judgments, as reported by Fleming et al., Gadjoz et al., and Wokke et al.
Why should decisions (or confidence judgments) consider action components? After all, choice accuracy can be optimized using sensory evidence, whereas action components may create maladaptive response biases. Perhaps the optimality principles that we consider during lab experiments are different from those embedded in our ancestral decision systems. Laboratory tasks focus mainly on one dimension of choice: its accuracy. During decisions “in the wild”, other factors such as costs and speed may become more prominent. Computational analyses reveal that embodied models afford both slower-but-more-accurate and faster-but-less-accurate decisions, hence spanning a wide portion of the speed-accuracy trade-off. Faster-but-less-accurate (or less costly) decisions are less supported by models lacking embodied mechanisms; but those may be particularly important during decisions “in the wild”.
Furthermore, and perhaps more importantly, perception, decision and action processes that we often strive to separate in the lab are typically intertwined “in the wild”. During a situated choice between multiple prey one can approach, initiating an action (approaching a specific prey) and even only an orienting response changes perception, decision, attention, memory and planning dynamics immediately. By going towards a specific prey, an animal collects more evidence relative to such prey, focuses more attention on it, lowers the sensitivity of information relative to other prey, remains committed to the current choice through feedback loops, and narrows the space of the other possible prey it could approach. Furthermore, the animal’s movements towards a gazelle also constitute a form of (offloaded) working memory of its current choice, potentially inducing additional commitment.
Note that these are exactly the same kind of factors that Anzulewicz and colleagues consider in their analysis, e.g., counting action or evidence or considering how it restricts the number of alternatives and modifies stimuli perception.
The journey we have summarized, from serial to embodied models of decision-making, contextualizes these factors within a cognitive architecture that puts situated aspects of choice at a premium. The importance of these and additional factors (e.g., agentive factors such as the fact that “I” am doing this action; proximal and distal goals) remains to be firmly established in future research (see also Rosenbaum and Fegghi, this issue); but they all make sense if one considers the challenges of embodied decisions that our ancient evolutionary ancestors had to face.
Main Psychonomics article considered in this post:
Anzulewicz, A., Hobot, J., Siedlecka, M., Wierzchoń, M. (2019). Bringing action into the picture. How action influences visual awareness. Attention, Perception, & Psychophysics. https://doi.org/10.3758/s13414-019-01781-w