Attention is a key part of cognition, and, therefore, it plays an important role across almost any context you can imagine. It’s critical for behaviors as diverse as air traffic control and keeping track of predators and influences outcomes with high relevance (such as academic performance). It is also sensitive to many factors such as caffeine intake, physical activity, emotional significance, and many others. And our coverage of attention-related research on the featured content site of the Psychonomic Society is quite popular.
With so many experimental manipulations and applied contexts where attention is a key variable, an easy to implement and reliable test that can be deployed in online environments is a very handy tool. Even more so in COVID-19 times, as the need of finding creative ways to continue research despite restrictions for in-person studies has proved invaluable.
Fernando Luna, Javier Roca, Elisa Martín-Arévalo, and Juan Lupiáñez report on the reliability of assessing attention and vigilance in the lab and online in their recent paper entitled, “Measuring attention and vigilance in the laboratory vs. online: The split-half reliability of the ANTI-Vea,” published in Behavior & Research Methods.
ANTI-Vea test
The authors report on the performance of the ANTI-Vea test. The ANTI-Vea test assesses classic attentional components and vigilance measures in online and laboratory settings. The test was designed taking into account an influential theory of attention that proposes attentional processes supported by the three following networks:
- The alerting network underlies phasic alertness (brief arousal increments) and vigilance (sustained attention over extended periods).
- The posterior network directs attentional orienting towards relevant stimuli.
- The anterior network modulates executive control processes for controlling attention to serve behaviors for long-term goals.
The ANTI-Vea assesses the independence and modulations of phasic alertness, orienting, and executive control, as well as two different subtypes of vigilance: executive vigilance and arousal vigilance. Executive vigilance occurs over long periods and require a controlled response. Arousal vigilance is a occurs quickly and without much of a controlled response executed.
To measure all these constructs in a single session, the ANTI-Vea combines a flanker task (where participants respond to the direction of a central arrow, ignoring the direction of neighboring arrows, which may point to the same or a different direction as the central arrow) with a spatial cueing paradigm (where a spatial cue may correctly predict the localization of an appearing target, incorrectly predict it, or the target may appear without being preceded with a cue), and an tone may (or may not) precede the appearance of the stimuli.
The flanker task measures executive control. The presence of a tone preceding the target measures phasic alertness. And the spatial cues (e.g., an asterisk appearing in different locations before the appearance of the target) measures orienting.
In addition, two types of infrequent trials appear throughout the task. If a central arrow appears displaced vertically, participants must press the spacebar (instead of reporting whether the arrow points to the left of the right). These trials evaluate executive vigilance. Lastly, if a countdown timer appears, any response is correct, insofar as it is performed fast, which assesses arousal vigilance. The following video shows the task in action.
[videopress 0hmK7b0Q]
The figure below summarizes the characteristics of all types of trials of the ANTI-Vea. In a are the stimuli sequence and timing for all trials except for the arousal vigilance ones (shown in b). In c are the examples of the type of visual cue conditions, and in d are the correct responses for each type of trial. The correct responses can be summarized as responding to the orientation of the central arrow ignoring the surrounding arrows except when the central arrow is displaced vertically (in which case the correct response is the space bar), or when a countdown timer appears, in which case any response is correct and should be executed as fast as possible. If you are in a hands-on mood, you can try the task yourself and download your data.
Comparing the ANTI-Vea in laboratory and online settings
More than 600 participants took part, about half in lab conditions and the other half online, to see whether the test is reliable in both settings.
As you can imagine, a complex task like this allows performing a wide range of analyses and looking at data from different angles. In this post, we’ll only illustrate some basic results of the experimental manipulations on reaction time (RT) measures, and insights on the reliability of both versions. The data have more to say by combining analyses of percentage of errors, nonparametric signal detection indices, and correlation analyses.
Attentional effects and reaction times
Reaction times allow capturing classic attentional effects in the trials where participants respond to the direction a central arrow points. The attentional effects include
- The warning signal effect where responses were faster in the tone than in the no tone condition.
- The visual cueing effect where responses were faster in the valid condition (i.e. when an asterisk appeared at the same location where the arrows appeared afterward) than in the invalid or no cue conditions.
- The congruency effect where responses were faster when the central and surrounding arrows pointed to the same rather than different directions.
The responses were faster in the standard laboratory setting than the online setting, but, critically, the three effects were clearly evident in both settings, as shown in the figure below.
In the trials where a countdown timer appeared, and participants should respond as fast as possible (i.e., trials assessing arousal vigilance), a significant increment in reaction times was observed across blocks. The same trend was observed in the standard and online settings, which implies that the task can detect decrements of arousal vigilance over time regardless of the setting, and despite the clear effect of the type of setting on the mean reaction time (with slower reaction times overall in the online than in the laboratory setting as shown below).
Paying attention to reliability
In addition to the similarity of the experimental effects shown above (and by additional ones described in depth in the paper), the authors computed split-half reliability indices for the standard and online versions for the different measures collected, including reaction times, error rates, and indices based on signal detection theory.
The results were clear: the reliability of the ANTI-Vea test is similar regardless of whether it is presented online or in its standard version, and for most measures, it achieves conventionally acceptable levels of reliability.
If you want to reliably measure attention online, you should be paying attention to the ANTI-Vea.
Featured Psychonomic Society article
Luna, F.G., Roca, J., Martín-Arévalo, E. & Lupiáñez, J. (2020) Measuring attention and vigilance in the laboratory vs. online: The split-half reliability of the ANTI-Vea. Behavior Research Methods. https://doi.org/10.3758/s13428-020-01483-4