Let’s set the stage with a series of images.
Each of these pictorial examples represents a different aspect of decision-making, which requires the ability to compare incoming stimuli quickly to behave appropriately. In the case of the flying stimuli in the first set of images, a human might be asked to categorize which stimulus is alive whereas a macaw might be asked to find the image most like itself (to my knowledge, this categorization has yet to be assessed in macaws ?). The second set of images might elicit a particular behavioral response in a dog, depending on how that dog interprets the ambiguous stimulus while a human may have trouble determining if the dog is playing or angry.
Categorization is a fundamental aspect of cognition for all animals, including humans. Whether a beaching killer whale is choosing between the adult sea lion (more kcals) vs. pup (easier prey) as it makes its move, or Superman is choosing to save Lois Lane (romantic love) vs. Martha Kent (familial love) from Lex Luther’s clutches, life and death decisions are dependent on the efficiency of decisions when faced with whatever stimuli requires it.
Many scientists attempted to quantify the moment of categorical decisions. Some researchers use reaction times, or how quickly a response is made to stimuli presented, such as in the case of the Implicit Association Test. In this test, the underlying, unconscious biases regarding various categories, like ethnicity, age, and political beliefs, are assessed. Other scientists use functional brain imaging techniques like fMRIs to follow the course of the process and categorization from sensation to cognition while still others use eye trackers to determine choices (see previous blog evaluating eye trackers). And finally, some researchers use the motor movements of individuals controlling a joystick or a mouse. The variety of measurement options is certainly diverse in terms of sensory systems.
Whatever the measurement device, the moment of making a decision is a highly sought-after goal, much like Superman’s attempt to rid the galaxy of kryptonite!
What’s in a decision?
A new metric was developed by David March and Lowell Gaertner (pictured below) to assess the time an individual initially categorizes a new stimulus correctly using a mouse-based motor movement. The results of the validation of this proposed metric were summarized in a paper published in the Psychonomic Society’s, Behavior Research Methods.
Take the task represented in the image below. Participants quickly categorize each face as calm or dangerous. Using a mouse to indicate their choice, the underlying cognitive-based decision process is assumed to be reflected in the motor movements and path trajectory of the mouse.
Panel a represents a hypothetical decision mouse track and panel b represents a real participant’s mouse track on a single trial.
The blue line in the figure above represents the mouse track while the solid black line represents the shortest distance between the starting point and endpoint, or a direct trajectory. Most important is the dotted black line(s), which represents the maximum deviation (MD) from the direct trajectory and the actual line.
For many studies, trials may range between 1000-1500 ms in length. Each trial is then divided into a number of time-based sample points (usually 10 ms). Depending on the length of a trial, the whole duration may be divided into equal time segments (e.g., 20 ms) to reduce the number of data points (such as from 100 to 50, if the duration of a trial was 1000 ms). This reduction process is called downsampling.
During two adjacent 10 ms time segments, the deviations identified by the dashed lines from the direct trajectory and the maximum inflection points of a person’s mouse trajectory are measured and averaged into one value for the 20 ms period. In previous studies of decision-making, these values are calculated using different techniques including Euclidean proximity, Euclidian distance, horizontal distance, or regression weights of the relationship between an x-location or x-y angle at each time point. Following these measurements, the data are then subjected to a series of statistical techniques to determine the point at which there is a significant difference between the target and the distractor in the trajectory.
March and Gaertner proposed that there were two major problems with this general approach. First, time cannot be directly estimated because it is dependent upon sample size with larger sample sizes having greater power to detect significant differences. Second, if multiple trials with the same stimulus are not available, individual differences cannot be assessed or controlled in a study.
To address these issues, the researchers took a two-step approach (and not the kind of two-stepping found in country dance halls) to estimate the time of initiating correct categorization. First, the Euclidean distance to the target and Euclidean distance to the distractor is measured for each time point, which the authors indicated was superior to either the vertical or horizontal distance. Second, these two differences are subtracted from each other, with the Euclidean distance from the target subtracted from the Euclidean distance of the distractor at each time point. Following these steps, the data were then subjected to two nonlinear regressions.
To visualize this process, look at the figure below. It shows two types of information. On the left are examples of three different mouse trajectories where the standardized values are plotted over time. On the right side of the figure, the top row illustrates a fairly direct trajectory while the bottom row illustrates an indecisive trajectory as shown by the multiple returns to each stimulus before a final decision is made. Moving down the right-hand column, the estimated sigmoid function (dashed black line) deviates more from the differences between the Euclidean distances of the actual trajectory path. These deviations were supported by increasingly poor-fitting models.
March and Gaertner were able to validate this technique using three different sets of data in which participants were asked to either identify a face that was either Asian, Black, or White as either angry or neutral in expression, or to categorize a face as Black or White based on the degree of typicality of facial racial-based features, or to identify a “politician’s” face as a man or woman after which they indicated the likelihood to which they would vote for each “politician.”
The results of their nonlinear regressions using two different types of distributions (Gompartz and Baranyi) produced models with maximum convergence (fit) as supported by high pseudo-R2 values (above .94) across all three sets of data. These results suggest that their technique was able to model a variety of types of mouse trajectories accurately.
The authors concluded that this technique estimates the time of initiating correct categorization (TICC) accurately and offers an opportunity to explore decision-making more precisely without the limitations of previous methods. They also summarised, “The TICC is an estimate of person-(or even trial-) level timing information cued to a specific stimulus and can be used as an outcome or predictor of other variables. Given such analytic flexibility, the TICC expands the possibilities for exploring mental processes via mouse-tracking.” However, as the authors mention themselves, this technique still has a way to go to truly be able to pinpoint the exact moment at which a decision is made during the cognitive process of categorization.
Much like Lois Lane’s desire to land a good news story, and learn who Superman truly was, while I would appreciate being able to identify the decision point in time, I am more intrigued by the possibility of using this technique to quantify decision-making in non-human animals. Many primates, including monkeys of different species, chimpanzees, and gorillas, have been taught to use a mouse or touchpad to indicate their choices in two-item, forced-choice categorization tasks (such as shown in the image below). Perhaps one day the metric developed by March and Gaertner could be used to determine the point at which these nonhuman animals make their correct decision when categorizing stimuli as being more or less.
Featured Psychonomic Society article
March, D. S., & Gaertner, L. (2021). A method for estimating the time of initiating correct categorization in mouse-tracking. Behavior Research Methods, 53(6), 2439-2449. https://doi.org/10.3758/s13428-021-01575-9