Have you ever played a video game where the first few levels are relatively easy and slow, but as you progress, the game becomes harder and faster? Years back, I played Tetris, Frogger, PacMan, Space Invaders, Centipede, etc. I can remember my 10-year-old self (or my 22-year-old graduate student self) working so hard to beat the level that my heart rate increased, sweat started dripping, my breaths came faster, and my fingers and hands cramped. These autonomic responses indicated my increased arousal or stress and meant that my sympathetic nervous system branch had kicked into gear to prepare for the possibility of fight or flight.
Why would the reflexive system activate while playing a video game? In a nutshell, many video games harness the excitement of the unexpected, including future rewards or significant losses. The well-honed stimuli stimulate the autonomic nervous system automatically.
What real-world, evolved contexts could a video game be simulating?
One context in which “expecting the unexpected” or “taking advantage of the current circumstances” are advantageous is during foraging. For many species, food is not guaranteed and is often unpredictable, dependent on the “whims” of nature and other competitors. Certain food patches need resources like water and specific minerals and temperatures to maintain their richness. Pollinators, like bees, do not share the latest updates on available patches with other pollinators like hummingbirds and butterflies, which leaves those creatures out to fly aimlessly.
Although this example may be slightly exaggerated, it does introduce the idea of sequential foraging. That is, deciding to stay with a currently available option or wait for an unknown future opportunity that could be better or not as good. More importantly, this decision is informed by past experiences, which includes the amount of effort needed to access the resources and previous usefulness or richness of the environment. A decision is dependent on optimizing the available knowledge relative to a risk assessment of a potential cost, with improvements in an environment learned faster than deteriorations.
Researchers Neil Dundon, Neil Garrett, Viktoriya Babenkol, Matt Cieslak, Nathaniel Daw, and Scott Grafton (pictured below) explored the influence of sequential foraging outcomes on the autonomic system of adult humans in a recent paper published by the Psychonomic Society’s Cognitive, Affective, & Behavioral Neuroscience journal. For this study, the researchers simulated sequential foraging using a current-day version of Space Invaders while monitoring the participants’ cardiac responses with an electrocardiogram (ECG) and an impedance cardiogram (ICG). These two measures recorded how fast the heart contracts and heart rate.
The figure below (a) also shows the procedure. As shown in the Capture and Release columns, the participant had to decide if the invader should be caught or released. A capture incurred a time cost and a fuel reward. A release produced no rewards and did not cost any time.
Play the video below to get a more concrete idea of their game. The green invader is a highly rewarding option (high reward, short capture time) while the purple invader is a low rewarding option (low reward, long capture time). There are two intermediate options (high reward, long capture time; yellow, and low reward, low capture time; brown). Participants had to choose whether they want to capture or release the invaders as they approach, one by one. The environment varied in richness by adjusting the number of green and blue invaders. The trick participants had to learn, is that when the environment is rich (lots of green invaders) they should wait for and only capture these invaders. However, when the environment is poor (lots of blue invaders) they should capture the intermediate options, as they have higher value in this context.
[videopress kTjVaLyD]
The researchers examined heart contractility (how quickly the heart contracts and pumps blood) as an immediate measure of changes in the autonomic system when participants’ were faced with several stressful outcomes.
Participants experienced four conditions with different degrees of profitability (represented in picture B above). The best outcome (greatest reward, green band) was the green invaders with a short capture time cost and maximum fuel. The worst outcome (smallest reward, brown band) were the purple saucers that had the longest capture time cost and minimum fuel reward. The two remaining outcomes were in the blue band and differed in capture time and reward value.To explore the influence of different environmental experiences during foraging, the researchers manipulated the order in which participants experienced two different environments. Some participants experienced a feast (boom) first, followed by famine (downturn), or BD. In contrast, others played the game under the opposite order, famine (downturn) followed by a feast (boom), or DB. Picture C illustrates the probability of the green/blue/brown outcomes during the boom and the downturn. That is, during the boom, more of the highest profitability space invaders were available for capture (green band) while during the downturn, more of the lowest profitability space invaders were available for capture (brown band).
Overall, the authors found evidence for the previously reported learning asymmetry in which contextual deterioration was learned more slowly than environmental improvement. This asymmetry in learning was observed when the reward value and capture time cost were altered in the middle range of profitability (blue band). This asymmetry can be seen in the blue bars of the two orders of environment contingencies, BD and DB.
As anticipated, the physiological measures supported the learning involved in maximizing one’s options during unpredictable conditions. Faster contractility was associated with higher captures regardless of the value of the reward, whereas slower heart rate was associated with capturing more low value space invaders.
One additional finding that was significant to understanding the relationship between sequential decisions and the autonomic system was that changes in the reward rate (the rate of reward harvested/sec) were associated with consistent changes in the sympathetic system measures. Basically, participants increased their capture rates more when a boom was experienced as opposed to a downturn (again, evidence of asymmetrical learning).
Following the behavioral and physiological outcomes, the authors tested various models of the expected outcomes. Across their models, they determined that the sympathetic branch is uniquely involved in learning and that they were predictive of optimal performance on the task. Thus, deteriorating environments are adapted to more slowly as negative knowledge slows down belief estimates and changes occur moment-to-moment.
How does our brain do all of this?
Perhaps one of the more fascinating aspects of this paper was the discussion of potential converging neurological evidence that underlies the optimization of decision-making under sequential contexts. It seems very likely that the behavioral, physiological, and model-based findings may be validated in the future with additional investigation of the neural circuitry underlying the sympathetic and parasympathetic branches of the autonomic system with a special emphasis on the anterior cingulate cortex (ACC).
The ACC plays an important role in learning, conflict monitoring, signaling of error and reward possibilities, foraging choices, and reward tracking. All of these processes are involved in the task that was tested in the current study and may prove to be rich foraging fields for future studies.
Featured Psychonomic Society article:
Dundon, N. M., Garrett, N., Babenko, V., Cieslak, M., Daw, N. D., & Grafton, S. T. (2020). Sympathetic involvement in time-constrained sequential foraging. Cognitive, Affective, & Behavioral Neuroscience, 20, 730-745. https://doi.org/10.3758/s13415-020-00799-0
