Is it likely to rain today? Do you think the Ravens will win their next game? Is now a good time to buy stock in that one tech company?

We make predictions like the ones above all the time. We do this using probability estimates. What is the probability of this event happening based on past events?

One theory argues that over time, people adapt their predictions in staircase-shaped patterns. That is, we adjust our probability estimates only occasionally in an abrupt shift—after considering the course of recent events. For example, let us consider fantasy sports betting. If a key player scores a touchdown or hits a home run, the bettor might quickly raise their probability estimate of winning the bet, making a large “step” in their judgment. After a few plays where the performance remains constant, they might not adjust their estimate until a significant new event happens, like another touchdown or a turnover. The probabilities don’t shift smoothly—but rather in clear, noticeable steps after key moments.

Several studies (Gallistel et al., 2014; Khaw et al., 2017; Ricci & Gallistel, 2017) observed these staircase-shaped responses in an estimation paradigm. The experiment asked participants to guess the proportion of blue rings in a box of blue and red rings. A single ring is drawn from the box, and then participants move a slider to estimate how many rings are blue (0% = no blue; 100% = all blue). In the next trial, participants had the option to lock in their previous estimate or make a new one. The experiment continued for 2,000 trials. These studies found staircase-shaped responses, and Khaw (2017) reported that “subjects adjust in discrete jumps rather than after each new piece of information.”

This finding goes against another theory that people continuously update their predictions based on associations between observed events. Mattias Forsgren, Peter Juslin, and Ronald van den Berg decided to investigate the robustness of the staircase-shaped pattern in a recent paper in Psychonomic Bulletin & Review.

In their first experiment, Forsgren and colleagues attempted to replicate the findings of previous studies. They used the “box of rings” paradigm described above. They did not inform participants about how the contents of the box could change throughout the experiment. In reality, the proportion of blue rings did change like a sine wave with a period of either 200 trials (short period) or 500 trials (long period). Participants were expected to track the change over time.

The authors failed to replicate the high level of accuracy reported in previous studies. They also found that some participants did not respond in staircase-shaped patterns. They did find that participants had better accuracy in the longer period of change (across 500 trials), but still did not have clear staircase curves.

In a second experiment, Forsgren and colleagues modified several aspects of the task. First, they manipulated the information that participants received about the task—telling them (or not) that the proportion of blue rings will shift over time. Second, they manipulated the effort level that it took to move the slider and make a prediction. One condition had the cursor stuck to the slider, so there was an equal amount of effort (low effort) to create a new prediction or keep the previous one. The other condition (high effort) had increased effort to change the prediction—that is, keeping the same prediction was easier than making a new one. Third, they incentivized accuracy to keep participants motivated to perform well. Their level of accuracy determined the value of a gift card they would receive for participating.

The authors were only able to replicate the accuracy of previous studies when the same conditions were used: when participants were informed that trials would change over time and when updating their predictions was more effortful. However, the step width (the feature that creates a staircase pattern) decreased when participants were given prior information about the task. This suggests that people actually track probabilities from trial to trial when given relevant information.

The authors claim that staircase-shaped probability learning is dependent on the participant’s knowledge about the task. They said:

“The primary takeaway from our findings is that previous conclusions were premature: accurate, stepwise updating is neither robust nor general in the way previously asserted. Another, more constructive, message is that taken together with prior research our findings suggest we should move beyond the associative learning versus hypothesis testing debate. We envision a theory in which individuals possess both the intuitive statistician’s capacity for data compression and the intuitive scientist’s capacity for forming and testing hypotheses about the generative process. The challenge for future research will be to understand how these capacities interact.”

Future work should continue to explore the nuances of real-world probability estimating and give us further details of when people “take the stairs.”

via GIPHY Source: https://www.afv.com/

Featured Psychonomic Society article

Forsgren, M., Juslin, P., & van den Berg, R. (2024). Further perceptions of probability: Accurate, stepwise updating is contingent on prior information about the task and the response mode. Psychonomic Bulletin & Review. https://doi.org/10.3758/s13423-024-02604-2