Treating errors as the signal rather than noise

Humans aren’t perfect – and neither are our brains. When solving problems, we often make mistakes or estimate an answer that’s good enough, but not exact. Typical methods to understanding cognition ignore these errors or treat them as random noise. But, in a recent article published in the Psychonomic Society journal Psychonomic Bulletin & Review, researchers propose that understanding errors can be key to understanding how people are solving problems.

Sundh Figure 1 — *Red neon error sign. Source: Florent Darrault (CC BY-SA 2.0)*

The researchers, Joakim Sundh, August Collsiӧӧ, Phillip Millroth and Peter Juslin (pictured below), build upon Egon Brunswik’s 1957’s definitions of intuitive and analytic cognitive processes. In Brunswik’s model, Intuition processes are noisy and inexact – errors are common, but typically small. The errors occur due to inherent noisiness of perception or estimation. In contrast, Analysis processes are exact and typically error-free. However, when errors do occur, they are large because they arise from misunderstanding the problem or using the wrong rule.

For example, imagine you are asked to calculate the area represented by the three different shapes in the image below. For one problem (A), you’re given the exact height and width of a right-angled triangle. For another problem (B), you’re trying to calculate the area of the same triangle, but you have to guess the height and width. Finally, in the third problem (C), both the exact shape and size is unknown.

Sundh Figure 3 — *Three types of area problems*

Assuming you remember how to calculate the area of a triangle (unlike me ?, apparently Area = (width*height)/2), your answers to problem type A are likely to be exactly correct. Any errors you make will be large and due to arithmetic errors. According to Brunkswik, you’d be using Analysis. However, when calculating the area of the irregular blob, your answers will be imprecise and noisy due to your use of Intuition processes. Finally, when solving the middle problems, you’d likely use a mix of both processes. Your estimation of the width and height will be noisy (Intuition), but your calculation of the area using those estimates will be exact (Analysis). And that’s exactly what you see in the actual error distributions from real participants below.

Sundh Figure 4 — *Histogram of the error distributions for area tasks*

Sundh and colleagues developed the Precise/Not Precise (PNP) model to identify these cognitive processes from error distributions. The model assumes that Intuition responses are sampled a homogeneous error distribution. In contrast, Analysis responses are sampled from a heterogeneous distribution (error-free and erroneous responses). The key parameter from the resulting model is λ or the probability that an error will occur. When using Intuition, λ = 1. All of the responses are imprecise. However, when using Analysis, λ will be small. Most answers are exactly correct.

It’s easiest to understand the model by walking through the responses of a few specific participants. Both of the participants below were solving the same type of problem. They were presented with a series of bets and asked what’s the most they’d be willing to pay to participate. For example, “The probability to win is .20 and if you win you receive 20 SEK (and 0 SEK otherwise): What is the highest price you would be willing to pay to partake in this lottery?” Economists often consider a “correct” response to be the one corresponding to the expected value of the bet or the average return if you were to play over and over again. For the problem above, the expected value is 4 SEK (20 * .2).

Participant 24’s responses are plotted below. Most of their answers fall exactly on the expected value line (long dashes). However, some responses are wildly off that function. If you conduct a standard linear regression, you’d conclude that the participant overestimates the expected value of each bet and that the overall standard deviation is 39.3.

Using the PNP model, the conclusion is that the majority of the participants’ responses are accurate, but 22% of the time they make an error (λ = .22) and the standard deviation of those errors is 92.6. Thus, the PNP model more accurately captures the participant’s actual strategy for solving the problems.

Sundh Figure 5 — *Responses from participant 24 in a willingness-to-pay task. The thicker dashed line is the expected value from each bet and the thinner dotted line is the best fitting regression line.*

In contrast, participant 84 uses an intuitive strategy rather than exactly calculating each expected value. Here, both the PNP model and a standard regression agree that the participant underestimates the value of each bet and does so with a consistent small amount of error (standard deviation = 4.67).

Sundh Figure 6 — “Responses from participant 84 in a willingness-to-pay task. The thicker dashed line is the expected value from each bet and the thinner dotted line is the best fitting regression line.”

Overall, the authors encourage other researchers to pay more attention to the shape of the error distributions. Errors aren’t only noise, they can be important signals for the type of cognitive processing.

Featured Psychonomic Society article

Sundh, J., Collsiöö, A., Millroth, P. & Juslin, P. (2021). Precise/not precise (PNP): A Brunswikian model that uses judgment error distributions to identify cognitive processes. Psychonomic Bulletin & Review, 28, 351–373. https://doi.org/10.3758/s13423-020-01805-9

Author

Lisa Fazio

Lisa Fazio’s research focuses on the human memory system and how our brains support and derail efforts to gain new, accurate knowledge. The same processes that support everyday learning can lead us astray when we are exposed to false information. Currently, an Assistant Professor in the Department of Psychology and Human Development at Vanderbilt University, her research focuses on how to mitigate the effects of reading false information and how to increase classroom learning.
View all posts