How stressed do you feel right now? To find out, take this assessment. I scored an 8, which puts me in the “you are stressed” category. I am not too surprised with my result given my own midlife status with two emerging adult children and too many responsibilities while attempting to return to “normal” in the midst of a pandemic that hasn’t quite progressed to an endemic status.

Covid-19 amplified stress around the world. While the global lockdowns allowed some people to slow down for a while (and reduce our human impact on the environment), most people reported experiencing high levels of stress. Many stressors included worrying about the virus itself, concern about family members, mitigation options (e.g., vaccines, masks, lockdowns), the economy, finances, technology, educational regression, spatial overlap between family life and work, and so on. Although much of the world is currently open and people are trying to return to business as usual, Covid-19 still has a grip on our lives.

No Reason to Stress!?

When the world started shutting down in early 2020, much of the ongoing research was also halted. Researchers who relied on people to conduct their studies found themselves at a crossroads – shift gears to study some other topic or take a hiatus from research. Despite the extra stress of the unknown, most researchers met these challenges with ingenuity and creative problem-solving. As academic classes shifted to online modalities, many researchers also shifted their platforms to conduct remote and online research projects associated with questions about the effects of the pandemic on almost every aspect of life – work-life balance, relationships, compliance with mandates, education, health, pets, technology, binge-watching, personality, etc. Some of these topics were highlighted in various posts by the Psychonomic Society Digital Editorial Staff and a recent podcast.

Mohammed Almazrouei, and his colleagues, Ruth Morgan and Itiel Dror (pictured below) were spurred to shift their research paradigms to an online forum.

In a paper recently published in the Psychonomic Society’s journal Behavior Research Methods, Almazrouei, Morgan, and Dror describe an experimental method used to induce stress in people participating in an online research environment.

Time’s Up!

Previous research involving stress tended to be conducted in person with most procedures manipulating some kind of stressful event experienced by participants, such as seeing emotionally disturbing images, experiencing uncomfortable physical conditions like ice baths, being forced to complete difficult/impossible tasks within a time limit, or being judged or critiqued by another. Working from a previously published meta-analysis in which social-evaluative threats (e.g., negative feedback on one’s performance) and uncontrollability (can’t avoid negative consequences) were the two most powerful manipulations that elicited stress responses reliably. Armed with this knowledge, Almazrouei and his colleagues developed a task that could induce stress within an online setting that allowed a researcher to be present virtually or not.

In their task, the researchers combined uncontrollable events with social-evaluative threats in an online study. Their research design is pictured in the graphic below.

Participants had to try to complete very difficult math or general knowledge questions in a very short time frame, such as 3-9 sec, depending on the question. Below are a couple of examples of “tricky” math problems.

Some participants received motivational messages both during and after the various questions while other participants received negative feedback, and control participants did not receive any feedback on their performance. Examples of the messages are in the table below.

In the figures above, the stress group showed higher levels of stress on average for both the trait anxiety measure (top panel) and the visual analog measure (bottom panel). Participants categorized as having high trait anxiety showed the highest levels of stress across all both conditions and on both measures (shown by the red points and 95% CI lines in both panels).

The researchers concluded that these patterns of results indicated that this procedure within an online platform produced valid online stress experiences that were measurable.  And, as the authors stated in a written interview, “It is important to understand how stress impacts decision-making because we often make important personal and professional decisions under stress. We now have a way to study these decisions with relative ease and with access to a wider and more diverse pool of participants.” To me, as a “sometimes” human researcher and a “more times” animal researcher, the more interesting component to this study were the controls that were implemented to ensure the validity of this procedure and its results.

Bots, bots, bots

When conducting experimental research, establishing true controls, experimental manipulations, manipulation checks, and quality control measures are challenging, but manageable, for in-person research studies because the participants are physically present and can be monitored directly. However, research conducted in online platforms asynchronously is more challenging in many ways, mainly because researchers not only have to ensure that participants are responding honestly, but that the participants are actually humans, and not bots.

Two recent studies discussed many of the difficulties associated with collecting data from an online forum or survey platform like MTurk, Prolific Academic, or Qualtrics. To address these issues, online research must implement attention checks, bot detectors, and manipulation checks without compromising the study. Almazrouei and his colleagues included several additional questions to achieve the best quality data. They included attention checks, bot checks, and filtered the acquired data for survey completion duration and patterns in responses. Examples of their exclusion criteria are found in the table below.

True or False?

As described earlier, online research must be able to differentiate valid data from false or artificial data. Thus, if a participant failed to follow the directions in the attention check question – “What color is the sky? Please select “purple” as an answer to make sure you are paying attention.”, then that participant was excluded from the final data set. The authors also included visual images of the various tasks and questions because they make it harder for participants to cheat or bots to answer the question reasonably.

As an animal researcher who occasionally enters into the human world of research, I appreciate the relative efficiency of conducting survey-based research, especially if it involves experimental manipulations. In my own experience, my online human study took 6 weeks to collect data while my animal research typically takes 1-2 years with a fast project. Of course, getting it out for publication takes the same amount of time, but I digress.

As Covid-19 has taught us, transitioning in-person activities to online experiences is the wave of the future, and while survey-based research may be faster to collect than other types of studies. I think I’ll stick with my animal research. At least, I only have to be as smart as a beluga or killer whale and not smarter than artificial intelligence or other humans!

Featured Psychonomic Society article

Almazrouei, M. A., Morgan, R. M., & Dror, I. E. (2022). A method to induce stress in human subjects in online research environments. Behavior Research Methods. https://doi.org/10.3758/s13428-022-01915-3